callonce is called multiple times with ignored scenario

[crazystick]

I think this is the same issue as #2292 which was closed without a resolution. I have a feature with an ignored scenario that's called by other scenarios in the same feature. Additionally in the Background I call a shared feature with callonce

I would expect when I run the feature, the shared feature is called exactly once. Instead it is called 3 times, once at the start and again each time the ignored scenario is called.

Runnable project here: https://github.com/crazystick/karate-issue-2292

Main feature:

Feature: sample karate test script
  for help, see: https://github.com/karatelabs/karate/wiki/IDE-Support

  Background:
    * callonce read('classpath:examples/shared/shared.feature')

  Scenario: Scenario 1
    * print 'Hello from Test 1 - Scenario 1'
    * call read('@generic_print') { message: 'Test 1 - Scenario 1' }

  Scenario: Scenario 2
    * print 'Hello from Test 1 - Scenario 2'
    * call read('@generic_print') { message: 'Test 1 - Scenario 2' }

  @generic_print @ignore
  Scenario: Generic Scenario
    * print 'Hello from Generic Scenario - called by ' + message

Shared feature:

Feature: Shared Feature

  Background:

  @scenario1
  Scenario: Shared Scenario
    * print "Hello from Shared Feature - Shared Scenario"

Output:

09:33:23.460 [main] INFO  com.intuit.karate - karate.env system property was: null 
09:33:23.487 [main] INFO  com.intuit.karate - >> lock acquired, begin callonce: read('classpath:examples/shared/shared.feature')
09:33:23.496 [main] INFO  com.intuit.karate - [print] Hello from Shared Feature - Shared Scenario 
09:33:23.498 [main] INFO  com.intuit.karate - << lock released, cached callonce: read('classpath:examples/shared/shared.feature')
09:33:23.505 [main] INFO  com.intuit.karate - [print] Hello from Test 1 - Scenario 1 
09:33:23.532 [main] INFO  c.intuit.karate.core.FeatureRuntime - classpath:examples/users/users.feature - call by tag at line 16: @generic_print
09:33:23.538 [main] INFO  com.intuit.karate - >> lock acquired, begin callonce: read('classpath:examples/shared/shared.feature')
09:33:23.543 [main] INFO  com.intuit.karate - [print] Hello from Shared Feature - Shared Scenario 
09:33:23.543 [main] INFO  com.intuit.karate - << lock released, cached callonce: read('classpath:examples/shared/shared.feature')
09:33:23.552 [main] INFO  com.intuit.karate - [print] Hello from Generic Scenario - called by Test 1 - Scenario 1 
09:33:23.563 [main] INFO  com.intuit.karate - karate.env system property was: null 
09:33:23.570 [main] INFO  com.intuit.karate - [print] Hello from Test 1 - Scenario 2 
09:33:23.571 [main] INFO  c.intuit.karate.core.FeatureRuntime - classpath:examples/users/users.feature - call by tag at line 16: @generic_print
09:33:23.576 [main] INFO  com.intuit.karate - >> lock acquired, begin callonce: read('classpath:examples/shared/shared.feature')
09:33:23.582 [main] INFO  com.intuit.karate - [print] Hello from Shared Feature - Shared Scenario 
09:33:23.583 [main] INFO  com.intuit.karate - << lock released, cached callonce: read('classpath:examples/shared/shared.feature')
09:33:23.587 [main] INFO  com.intuit.karate - [print] Hello from Generic Scenario - called by Test 1 - Scenario 2 
---------------------------------------------------------
feature: classpath:examples/users/users.feature
scenarios:  2 | passed:  2 | failed:  0 | time: 0.0949
---------------------------------------------------------

Karate version: 1.5.0
======================================================
elapsed:   1.29 | threads:    1 | thread time: 0.10 
features:     1 | skipped:    0 | efficiency: 0.07
scenarios:    2 | passed:     2 | failed: 0
======================================================

HTML report: (paste into browser to view) | Karate version: 1.5.0
file:///xxxxx/target/karate-reports/karate-summary.html
===================================================================

[ptrthomas]

@crazystick two things:

• things like callonce make sense when they return data (ideally JSON) that will be cached. try doing * def result = callonce read('shared.feature') and see if that makes a difference
• @ignore does not apply when you use call, I thought this was documented somewhere, perhaps it needs to be

[crazystick]

1. In my actual code, Shared Scenario is responsible for setting a cookie which other requests need to authenticate. There isn't really any data returned. I did try karate.callSingle but that doesn't set cookies (or not without some complicated hacking).
2. Yes, that's what I intended, the whole point is that Generic Scenario is called by other Scenarios but not run otherwise. Shared Feature is also not called from a runner, only used as a helper.

Edit: I tried adding def result = in front and it still runs the shared feature multiple times, I think I tried that in my actual code and in addition to not solving the problem it also didn't set the cookie anymore either

[ptrthomas]

@crazystick until this is fixed (no ETA, PRs welcome) - here's the workaround I suggest. get the data needed for the cookie setting in the callonce or callSingle, then do a normal call and set cookies - which will not be a time-consuming operation at all. this is the pattern most teams use, also see: #2539