Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parametric: adds incomplete_test decorator #3551

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

mabdinur
Copy link
Contributor

@mabdinur mabdinur commented Nov 25, 2024

Motivation

Adds a new decorator to distinguish between whether a test is failing because a feature is not supported or if the test relies on an unimplemented/incomplete interface.

For example:

  • The ruby tracer supports setting resource names on spans but the ruby parametric app does not support the trace/span/set_resource endpoint. With this proposal all tests that rely on this unsupported endpoint will be annotated with incomplete_test.

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@mabdinur mabdinur force-pushed the munir/add-incomplete-test-annotation branch from 579c10a to caaafcf Compare November 25, 2024 21:55
@mabdinur mabdinur force-pushed the munir/add-incomplete-test-annotation branch from caaafcf to b237c53 Compare November 25, 2024 21:55
@mabdinur mabdinur changed the title parametric: add incomplete_test_app annotation parametric: adds test_inconsistency decorator Nov 25, 2024
@mabdinur mabdinur changed the title parametric: adds test_inconsistency decorator parametric: adds incomplete_test decorator Nov 25, 2024
@mabdinur mabdinur force-pushed the munir/add-incomplete-test-annotation branch from dbe12ac to d2747ae Compare November 25, 2024 22:44
@mabdinur mabdinur force-pushed the munir/add-incomplete-test-annotation branch from ebb7814 to b2759c3 Compare November 25, 2024 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant