Cypress tests randomly hang in some machine when running in parallel using cypress-cloud via Jenkins CI

[bilarallen]

In the previous weeks, we were experiencing smooth automated runs with Cypress using cypress-cloud and via Jenkins CI. Until such time we were experiencing weird behavior where the test randomly stuck in some VMs when we are running the test in parallel using cypress cloud. Our setup goes like this:

1. We are using Jenkins CI
2. Our setup is using 5 VMs using AWS ec2 fleet instance for parallel run
3. Orchestration batch size for e2e tests is set to 4 in currents.config.js

Cypress cloud run result:

Machine A console log where the test hangs (still tests are running):

Machine B console log where the test also hangs (still tests are running)

We can't determine whether this is a problem with the CI Machines or Cypress-cloud as this is randomly happening and we can't extract the logs for each test since we are currently running 113 tests and our console logs might be bombarded if we do so, not to mention the maximum test runs we have on our license.

Also, this will cost as a lot in AWS since running ec2 fleet machines costs us per hour of usage so we also want to resolve this issue. Hope someone can help us in regards to this. Thanks! 🙇‍♂️

[agoldis]

@bilarallen try setting a timeout on Jenkins / AWS - can save you resources. As for non-responsive tests - there can be various reasons. Please try running the tests with debug mode enabled - it should help with identifying why the tests aren't progressing:

https://currents.dev/readme/integration-with-cypress/troubleshooting

[bilarallen]

Hi @agoldis tests running in a specific VM machine during parallel run suddenly hang again and caused the run to timeout

Here's the envi info of the VM machine

+ npx envinfo --system --binaries --browsers --npmPackages --duplicates --npmGlobalPackages
npx: installed 1 in 1.364s

  System:
    OS: Linux 5.15 Debian GNU/Linux 10 (buster) 10 (buster)
    CPU: (4) x64 Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
    Memory: 11.31 GB / 15.44 GB
    Container: Yes
    Shell: 5.0.3 - /bin/bash
  Binaries:
    Node: 14.16.0 - /usr/local/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.14.11 - /usr/local/bin/npm
  npmPackages:
    @babel/eslint-parser: ^7.21.8 => 7.21.8 
    @currents/cli: ^4.0.3 => 4.0.3 
    @cypress/angular:  0.0.0-development 
    @cypress/mount-utils:  0.0.0-development 
    @cypress/react:  0.0.0-development 
    @cypress/react18:  0.0.0-development 
    @cypress/svelte:  0.0.0-development 
    @cypress/vue:  0.0.0-development 
    @cypress/vue2:  0.0.0-development 
    @cypress/webpack-batteries-included-preprocessor: ^2.4.0 => 2.4.0 
    @cypress/webpack-preprocessor: ^5.17.0 => 5.17.0 
    @faker-js/faker: ^7.6.0 => 7.6.0 
    babel-eslint: ^10.1.0 => 10.1.0 
    cypress: ^12.11.0 => 12.11.0 
    cypress-cloud: ^1.7.3 => 1.7.3 
    cypress-network-idle: ^1.14.2 => 1.14.2 
    cypress-real-events: ^1.7.6 => 1.7.6 
    cypress-recurse: ^1.27.0 => 1.27.0 
    eslint: ^8.39.0 => 8.39.0 
    moment: ^2.29.4 => 2.29.4 
    numeral: ^2.0.6 => 2.0.6 
    swagger-client: ^3.18.5 => 3.18.5 
    uuid: ^9.0.0 => 9.0.0 (8.3.2)
    xvfb: ^0.4.0 => 0.4.0 
  npmGlobalPackages:
    npm: 6.14.11
    yarn: 1.22.10

Here's the debug log, I have put it in a text file since the log is very long:
Cypress_Cloud_Discussion_127_Debug_Logs.txt

Here's the screenshot of the run:

Hoping for some help and an answer for this, thanks!

[agoldis]

@bilarallen thanks for sending the details, looking into!

P.S. you'd better use the in-app support chat for faster response as Currents customer

[bilarallen]

Oh, I forgot. Yeah, I will chat using in-app support next time. Looking forward to a solution to this problem. Thanks @agoldis