[DX-1591] Integrate Tyk Operator Endpoint Level Features (Draft)

[dcs3spp]

User description

For internal users - Please add a Jira DX PR ticket to the subject!

DX-1591

---

Preview Link

preview - allow list

preview - block list

preview - circuit breaker

preview - do not track

preview - endpoint cache

preview - enforced timeout

preview - ignore

preview - internal endpoint

preview - mock response

preview - request body transform

preview - request header transform

preview - request method transform

preview - request size limit

preview - response body transform

preview - response header transform

preview - url rewrite

preview - validate request

preview - virtual endpoint

Description

Integrate Endpoint Level Tyk Operator features, adding examples for endpoint level features.

Question for reviewers

• Is threshold in the example invalid for Dashboard API and is the enforced timeout needed? Is there no action needed for this?

• Should there be an advanced trigger example in this page also? Currently only have examples for URL Rewrite triggers in Tyk Operator GHub internal looping? See here It is marked as unsupported feature but internal looping examples uses it

• What is act_on field in example and should it be removed from the examples it appears in if not needed? Answered in comments, it should be retained.

• Should there be corresponding API level examples for Request size limit
 & Response header transforms?
  Added in API Global features PR

• Is there a typo in path of virtual section in the yaml example, should the path be /virtual instead of virtual?
  Fixed

• In cache options configuration object in caching example there are options to configure the cache_response_codes and cache headers (commented). However these are not present in simple caching options for GW. Missing in GW docs?

Screenshots (if appropriate)

Checklist

• I have added a preview link to the PR description.
• I have reviewed the suggestions made by our AI (PR Agent) and updated them accordingly (spelling errors, rephrasing, etc.)
• I have reviewed the guidelines for contributing to this repository.
• I have read the technical guidelines for contributing to this repository.
• Make sure you have started your change off our latest master.
• I labeled the PR

---

PR Type

documentation

---

Description

• Added detailed sections for configuring various middleware features in Tyk Operator, including Allow List, Block List, Circuit Breaker, and more.
• Provided YAML configuration examples for each middleware feature to guide users in setting up Tyk Operator.
• Updated compatibility table in API definition reference to include direct links to middleware configuration examples.
• Improved documentation clarity and consistency across middleware configuration guides.

---

Changes walkthrough 📝

Relevant files

Documentation

19 files allow-list-tyk-classic.md Add Allow List Configuration for Tyk Operator                        tyk-docs/content/product-stack/tyk-gateway/middleware/allow-list-tyk-classic.md Added section for configuring Allow List in Tyk Operator. Updated JSON code block formatting. Added reference link for Tyk Operator configuration. +53/-2    block-list-tyk-classic.md Add Block List Configuration for Tyk Operator                        tyk-docs/content/product-stack/tyk-gateway/middleware/block-list-tyk-classic.md Added section for configuring Block List in Tyk Operator. Updated JSON code block formatting. Added reference link for Tyk Operator configuration. +56/-1    circuit-breaker-tyk-classic.md Add Circuit Breaker Configuration for Tyk Operator              tyk-docs/content/product-stack/tyk-gateway/middleware/circuit-breaker-tyk-classic.md Added section for configuring Circuit Breaker in Tyk Operator. Provided YAML example for Tyk Operator configuration. +50/-2    do-not-track-tyk-classic.md Add Do Not Track Configuration for Tyk Operator                    tyk-docs/content/product-stack/tyk-gateway/middleware/do-not-track-tyk-classic.md Added section for configuring Do Not Track middleware in Tyk Operator. Provided YAML example for Tyk Operator configuration. +44/-1    endpoint-cache-tyk-classic.md Add Endpoint Cache Configuration for Tyk Operator                tyk-docs/content/product-stack/tyk-gateway/middleware/endpoint-cache-tyk-classic.md Added section for configuring Endpoint Cache in Tyk Operator. Provided examples for simple and advanced caching configurations. +103/-1  enforced-timeout-tyk-classic.md Add Enforced Timeout Configuration for Tyk Operator            tyk-docs/content/product-stack/tyk-gateway/middleware/enforced-timeout-tyk-classic.md Added section for configuring Enforced Timeout in Tyk Operator. Provided YAML example for Tyk Operator configuration. +62/-1    ignore-tyk-classic.md Add Ignore Authentication Configuration for Tyk Operator  tyk-docs/content/product-stack/tyk-gateway/middleware/ignore-tyk-classic.md Added section for configuring Ignore Authentication middleware in Tyk Operator. Provided YAML example for Tyk Operator configuration. +48/-1    internal-endpoint-tyk-classic.md Add Internal Endpoint Configuration for Tyk Operator          tyk-docs/content/product-stack/tyk-gateway/middleware/internal-endpoint-tyk-classic.md Added section for configuring Internal Endpoint middleware in Tyk Operator. Provided YAML example for Tyk Operator configuration. +40/-1    mock-response-tyk-classic.md Add Mock Response Configuration for Tyk Operator                  tyk-docs/content/product-stack/tyk-gateway/middleware/mock-response-tyk-classic.md Added section for configuring Mock Response middleware in Tyk Operator. Provided YAML example for Tyk Operator configuration. +52/-1    request-body-tyk-classic.md Add Request Body Transform Configuration for Tyk Operator tyk-docs/content/product-stack/tyk-gateway/middleware/request-body-tyk-classic.md Added section for configuring Request Body Transform in Tyk Operator. Provided YAML example for Tyk Operator configuration. +75/-1    request-header-tyk-classic.md Add Request Header Transform Configuration for Tyk Operator tyk-docs/content/product-stack/tyk-gateway/middleware/request-header-tyk-classic.md Added section for configuring Request Header Transform in Tyk Operator. Provided YAML example for Tyk Operator configuration. +84/-4    request-method-tyk-classic.md Add Request Method Transform Configuration for Tyk Operator tyk-docs/content/product-stack/tyk-gateway/middleware/request-method-tyk-classic.md Added section for configuring Request Method Transform in Tyk Operator. Provided YAML example for Tyk Operator configuration. +46/-2    request-size-limit-tyk-classic.md Add Request Size Limit Configuration for Tyk Operator        tyk-docs/content/product-stack/tyk-gateway/middleware/request-size-limit-tyk-classic.md Added section for configuring Request Size Limit in Tyk Operator. Provided examples for API and endpoint-level configurations. +82/-4    response-body-tyk-classic.md Add Response Body Transform Configuration for Tyk Operator tyk-docs/content/product-stack/tyk-gateway/middleware/response-body-tyk-classic.md Added section for configuring Response Body Transform in Tyk Operator. Provided YAML example for Tyk Operator configuration. +144/-1  response-header-tyk-classic.md Add Response Header Transform Configuration for Tyk Operator tyk-docs/content/product-stack/tyk-gateway/middleware/response-header-tyk-classic.md Added section for configuring Response Header Transform in Tyk Operator. Provided examples for API and endpoint-level configurations. +204/-3  url-rewrite-tyk-classic.md Add URL Rewriter Configuration for Tyk Operator                    tyk-docs/content/product-stack/tyk-gateway/middleware/url-rewrite-tyk-classic.md Added section for configuring URL Rewriter in Tyk Operator. Provided YAML example for Tyk Operator configuration. +45/-2    validate-request-tyk-classic.md Add Validate Request Configuration for Tyk Operator            tyk-docs/content/product-stack/tyk-gateway/middleware/validate-request-tyk-classic.md Added section for configuring Validate Request middleware in Tyk Operator. Provided YAML example for Tyk Operator configuration. +56/-1    virtual-endpoint-tyk-classic.md Add Virtual Endpoint Configuration for Tyk Operator            tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md Added section for configuring Virtual Endpoint in Tyk Operator. Provided YAML example for Tyk Operator configuration. +97/-1    api-definition.md Update Middleware Feature Links and Descriptions                  tyk-docs/content/product-stack/tyk-operator/reference/api-definition.md Updated links in the compatibility table for middleware features. Improved clarity of feature descriptions. +20/-20

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[buraksekili]

lgtm! thank you!!

[caroltyk]

@dcs3spp, added previous response from @buraksekili here too

Question for reviewers

• Is threshold in the example invalid for Dashboard API and is the enforced timeout needed?

threshold is valid for Dashboard API as well. When I created this API Definition CR and i am able to see that threshold is set correctly on the dashboard. For the enforced timeout, i believe we might create another sample for it. The current sample only configures simple circuit_breaker for a reference. @buraksekili why do we need a new sample? Is the current one working?

• Should there be an advanced trigger example in this page also? Currently only have examples for URL Rewrite triggers in Tyk Operator GHub internal looping?

Advanced trigger is marked as not supported

• What is act_on field in example and should it be removed from the examples it appears in if not needed?

act_on is part of the API Definition (reference, here) and to be honest i am not sure what feature it enables. So, please keep it in the API Definition CRD as well.

• Should there be corresponding API level examples for Request size limit
 & Response header transforms?

I've tested with following manifest:

API level Request size limit:

apiVersion: tyk.tyk.io/v1alpha1
kind: ApiDefinition
metadata:
  name: httpbin-global-limit
spec:
  name: httpbin-global-limit
  use_keyless: true
  protocol: http
  active: true
  proxy:
    target_url: http://httpbin.org
    listen_path: /httpbin-global-limit
    strip_listen_path: true
  version_data:
    default_version: Default
    not_versioned: true
    versions:
      Default:
        global_size_limit: 5
        name: Default

API Response Header Transform

apiVersion: tyk.tyk.io/v1alpha1
kind: ApiDefinition
metadata:
  name: httpbin-global-header
spec:
  name: httpbin-global-header
  use_keyless: true
  protocol: http
  active: true
  proxy:
    target_url: http://httpbin.org
    listen_path: /httpbin-global-header
    strip_listen_path: true
  version_data:
    default_version: Default
    not_versioned: true
    versions:
      Default:
        name: Default
        use_extended_paths: true
        paths:
          black_list: []
          ignored: []
          white_list: []
        global_response_headers:
          X-Static: foobar
          X-Request-ID: "$tyk_context.request_id"
          X-User-ID: "$tyk_meta.uid"
        global_response_headers_remove:
        - X-Secret

• Is there a typo in path of virtual section in the yaml example, should the path be /virtual instead of virtual?

@komalsukhani can you confirm?

• In cache options configuration object in caching example there are options to configure the cache_response_codes and cache headers (commented). However these are not present in simple caching options for GW.

It might be missing in original gateway docs.

[buraksekili]

@caroltyk for enforced t/o, maybe we can update CR according to API Designer example so that all samples are aligned across platforms - we do not need to create a new one though. Our current sample combines enforced timeouts with circuit breakers which is still valid for enforced t/o

[dcs3spp]

Thanks @caroltyk

I have integrated the response header transform and request size limit examples.

For the response header transform should there also be an example for GW versions < 5.3.0 similar to the endpoint level examples on the same page?

[dcs3spp]

@caroltyk Advanced triggers for URL rewrites have been used in the internal looping example, in this section for the API entry point.

Should the status of Advanced triggers for URL rewrites be changed in the table to mark them as supported and an example added to this page or alternatively a link added to internal looping example?

[github-actions]

PR Reviewer Guide 🔍

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Key issues to review Documentation Clarity The new section added for configuring the allow list in Tyk Classic API Definition lacks detailed examples or code snippets which might be necessary for clear understanding. Documentation Clarity Similar to the allow list, the block list documentation could benefit from more detailed examples or clearer explanation of the configuration steps. Missing Details The section on configuring the Circuit Breaker in Tyk Operator is missing specific examples of how to set up the circuit breaker properties. Inconsistency The documentation introduces an "Advanced endpoint cache" section but does not adequately differentiate it from the simple cache setup.

[github-actions]

PR Code Suggestions ✨

Category	Suggestion	Score
Enhancement	Update terminology in code examples to match the modernized term "allow list" Consider using consistent terminology for "allow list" throughout the document to avoid confusion. The term "white_list" is used in the code example, which is inconsistent with the modernized term "allow list" used in the text. It's recommended to update the code snippets to reflect this terminology change for consistency and clarity. tyk-docs/content/product-stack/tyk-gateway/middleware/allow-list-tyk-classic.md [16] -To enable and configure the allow list you must add a new `white_list` object to the `extended_paths` section of your API definition. +To enable and configure the allow list you must add a new `allow_list` object to the `extended_paths` section of your API definition. Suggestion importance[1-10]: 8 Why: The suggestion addresses a significant inconsistency in terminology between the code and the documentation, which can lead to confusion. Updating the terminology to "allow list" improves clarity and aligns with modern practices.	8
	Update terminology in code examples to match the modernized term "block list" To improve clarity and maintain consistency, replace the term "black_list" with "block_list" in the code example, aligning with the modernized terminology used in the text. tyk-docs/content/product-stack/tyk-gateway/middleware/block-list-tyk-classic.md [18] -To enable and configure the block list you must add a new `black_list` object to the `extended_paths` section of your API definition. +To enable and configure the block list you must add a new `block_list` object to the `extended_paths` section of your API definition. Suggestion importance[1-10]: 8 Why: This suggestion addresses a terminology inconsistency, which is important for clarity and alignment with modernized language. It improves the documentation by making it more consistent and understandable.	8
	Add an actual example for API level configuration in Tyk Operator Replace the placeholder comment with an actual example for configuring the API level in Tyk Operator. This will provide a clear guide for users on how to implement configurations. tyk-docs/content/product-stack/tyk-gateway/middleware/request-header-tyk-classic.md [148] -<!-- need an example here --> +```yaml +apiVersion: tyk.tyk.io/v1alpha1 +kind: ApiDefinition +metadata: + name: example-api +spec: + name: example-api + use_keyless: true + protocol: http + active: true + proxy: + target_url: http://example.com + listen_path: /example-path + strip_listen_path: true + version_data: + default_version: Default + not_versioned: true + versions: + Default: + name: Default + use_extended_paths: true + extended_paths: + transform_headers: + - add_headers: + X-Example-Header: "Example Value" + - delete_headers: + - "X-Remove-Header" +``` Suggestion importance[1-10]: 8 Why: Providing a concrete example enhances the documentation by offering practical guidance, which is beneficial for users implementing configurations.	8
	Provide a concrete example for setting a request size limit for a specific API in Tyk Operator Replace the placeholder comment with an actual example for configuring the request size limit for a specific API in Tyk Operator. This will help users understand how to apply these settings effectively. tyk-docs/content/product-stack/tyk-gateway/middleware/request-size-limit-tyk-classic.md [95] -<!-- Need an example --> +```yaml +apiVersion: tyk.tyk.io/v1alpha1 +kind: ApiDefinition +metadata: + name: example-api-size-limit +spec: + name: example-api-size-limit + use_keyless: true + protocol: http + active: true + proxy: + target_url: http://example.com + listen_path: /example-size-limit + strip_listen_path: true + version_data: + default_version: Default + not_versioned: true + versions: + Default: + global_size_limit: 1024 # Limit in bytes +``` Suggestion importance[1-10]: 8 Why: Including a specific example helps users understand how to apply request size limits effectively, improving the usability of the documentation.	8
	Provide a specific example for API level transformation in the Tyk Operator documentation Add a specific example for the API level transformation in the Tyk Operator section to guide users on configuring response headers effectively. tyk-docs/content/product-stack/tyk-gateway/middleware/response-header-tyk-classic.md [195] -<!-- need an example here --> +```yaml +apiVersion: tyk.tyk.io/v1alpha1 +kind: ApiDefinition +metadata: + name: example-api-response-header +spec: + name: example-api-response-header + use_keyless: true + protocol: http + active: true + proxy: + target_url: http://example.com + listen_path: /example-response-header + strip_listen_path: true + version_data: + default_version: Default + not_versioned: true + versions: + Default: + global_response_headers: + X-Example-Header: "Example Value" + global_response_headers_remove: + - "X-Remove-Header" +``` Suggestion importance[1-10]: 8 Why: Adding an example for API level transformation aids users in configuring response headers, making the documentation more practical and user-friendly.	8
	Add a brief description or example to the Tyk Operator configuration section Consider adding a brief description or example of how to configure the middleware in the Tyk Operator section to improve clarity and provide immediate guidance for users unfamiliar with the process. tyk-docs/content/product-stack/tyk-gateway/middleware/virtual-endpoint-tyk-classic.md [19] -If you're using Tyk Operator then check out the [configuring the middleware in Tyk Operator](#tyk-operator) section below. +If you're using Tyk Operator, see the [configuring the middleware in Tyk Operator](#tyk-operator) section below for a step-by-step guide and examples on setting up your middleware. Suggestion importance[1-10]: 7 Why: The suggestion enhances clarity by providing additional guidance for users unfamiliar with the process, which improves the documentation's usability.	7
	Correct a typo in the URL rewriter configuration example to enhance clarity Correct the typo in the URL rewriter configuration example to ensure clarity and accuracy in the documentation. tyk-docs/content/product-stack/tyk-gateway/middleware/url-rewrite-tyk-classic.md [37] -In this example the basic trigger has been configured to match the path for a request to the `GET /books/author` endpoint against the pure regex `(\w+)/(\w+)`. This is looking for two word groups in the path which, if found, will store the first string (`books`) in context variable `$1` and the second (`author`) in `$2`. The request (target) URL will then be rewritten to `library/service?value1=books&value2=author` ready for processing by the next middleware in the chain. +In this example, the basic trigger has been configured to match the path for a request to the `GET /books/author` endpoint against the regex `(\w+)/(\w+)`. This looks for two word groups in the path which, if found, will store the first string (`books`) in the context variable `$1` and the second (`author`) in `$2`. The request (target) URL will then be rewritten to `library/service?value1=books&value2=author`, ready for processing by the next middleware in the chain. Suggestion importance[1-10]: 5 Why: Correcting typos improves the clarity and professionalism of the documentation, although it is a minor enhancement.	5
Best practice	Ensure exact path matching by adding a termination character It's recommended to add a termination character ('$') at the end of the endpoint path in the JSON example to ensure that the path is matched exactly as intended without allowing additional characters at the end. tyk-docs/content/product-stack/tyk-gateway/middleware/allow-list-tyk-classic.md [131] -"path": "/get" +"path": "/get$" Suggestion importance[1-10]: 7 Why: Adding a termination character is a best practice for ensuring exact path matching, which can prevent unintended behavior. This suggestion enhances the robustness of the configuration.	7