You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
marathon/html/pools/omni.html page says that test failed, but when I click on failed test and go to test details, report says that test succeeded. Individual test report page is the following: marathon/html/pools/omni/127.0.0.1-5037-emulator-5554/com.example.MyTest.html
I execute single test classMyTest.html which contains 7 different tests. 5 passes, but 2 tests fail. I noticed that on tests list page (marathon/html/pools/omni.html) every test references the same marathon/html/pools/omni/127.0.0.1-5037-emulator-5554/com.example.MyTest.html file and marathon/html/pools/omni/127.0.0.1-5037-emulator-5554 folder contains only one file. This file contains only one entry about the last executed test.
I suspect that it might be related to count sharding usage
shardingStrategy:
type: "count"count: 5
To Reproduce
Configuration file
name: "UI tests"outputDir: "../../build/reports/marathon"screenRecordingPolicy: "ON_FAILURE"deviceInitializationTimeoutMillis: 180000debug: trueshardingStrategy:
type: "count"count: 5executionStrategy:
mode: ALL_SUCCESSuncompletedTestRetryQuota: 3vendorConfiguration:
type: "Android"# Required for parameterized tests to be run correctly. More info in https://github.com/MarathonLabs/marathon/blob/develop/docs/docs/android/configure.md#test-parsertestParserConfiguration:
type: "remote"instrumentationArgs:
listener: "com.malinskiy.adam.junit4.android.listener.TestAnnotationProducer"notAnnotation: "com.my.package.ScreenshotTestSuite"instrumentationArgs:
debug: "false"disableWindowAnimation: trueautoGrantPermission: truescreenRecordConfiguration:
preferableRecorderType: "screenshot"applicationPmClear: truetestApplicationPmClear: truewaitForDevicesTimeoutMillis: 60000outputs:
- testApplication: "../../integration_tests/build/outputs/apk/androidTest/debug/integration_tests-debug-androidTest.apk"outputConfiguration:
maxPath: 1024filteringConfiguration:
allowlist:
- type: "simple-class-name"values:
- "MyFlakyTest"
Expected behavior
Test detail report page should also report failure with all relevant information (screenshot, logs, etc).
Logs and reports
Sorry, I cannot share full report as it might contain private information. But I'm more than happy to answer any additional question.
The text was updated successfully, but these errors were encountered:
Describe the bug
marathon/html/pools/omni.html
page says that test failed, but when I click on failed test and go to test details, report says that test succeeded. Individual test report page is the following:marathon/html/pools/omni/127.0.0.1-5037-emulator-5554/com.example.MyTest.html
I execute single test class
MyTest.html
which contains 7 different tests. 5 passes, but 2 tests fail. I noticed that on tests list page (marathon/html/pools/omni.html
) every test references the samemarathon/html/pools/omni/127.0.0.1-5037-emulator-5554/com.example.MyTest.html
file andmarathon/html/pools/omni/127.0.0.1-5037-emulator-5554
folder contains only one file. This file contains only one entry about the last executed test.I suspect that it might be related to count sharding usage
To Reproduce
Configuration file
Expected behavior
Test detail report page should also report failure with all relevant information (screenshot, logs, etc).
Logs and reports
Sorry, I cannot share full report as it might contain private information. But I'm more than happy to answer any additional question.
The text was updated successfully, but these errors were encountered: