[DX-1590] Tyk Operator : API Level Global Features (Draft)

[dcs3spp]

User description

For internal users - Please add a Jira DX PR ticket to the subject!

DX-1590

---

Preview Link

preview - link to internal looping from URLRewrite page
preview - internal looping

preview - request header transformations
preview - response header transformations
preview - segment tags
preview - CORS
preview - gRPC
preview - Javascript
preview - Custom Plugin Analytics
preview - Round Robin Load Balancing

preview - Do not track middleware
preview - Analytics tagging

Description

Add examples for API level global features. The PR includes all examples except for Config Data which had target page marked as missing. The config data example is included in virtual endpoints.

Question for reviewers:

• Is there a working example for Tyk Operator for configuring gRPC to use remote bundles? If plugin bundles are not supported in Tyk Operator then content in PR should be updated to state this. There is an example in this PR for gRPC with custom middleware manifest in API definition but not an example that links to the bundled zip file containing the manifest.

• Should the following API global features be included with examples?

  • Analytics API tagging - Done example added ready for review
  • Do not track analytics (per API) - Done example added ready for review
  • Custom plugins:
    • Go
    • Lua
    • Python

Screenshots (if appropriate)

Checklist

• I have added a preview link to the PR description.
• I have reviewed the suggestions made by our AI (PR Agent) and updated them accordingly (spelling errors, rephrasing, etc.)
• I have reviewed the guidelines for contributing to this repository.
• I have read the technical guidelines for contributing to this repository.
• Make sure you have started your change off our latest master.
• I labeled the PR

---

PR Type

Documentation

---

Description

• Added extensive documentation for Tyk Operator, including examples for API shard tagging, global rate limiting, load balancing, and CORS configuration.
• Introduced new sections on configuring custom plugins and internal looping with Tyk Operator, providing detailed examples and YAML configurations.
• Enhanced existing documentation with Tyk Operator examples for analytics plugins, gRPC plugin hooks, request and response header transformations, and middleware configuration.
• Updated menu and reference links to include new documentation entries and ensure consistency.

---

Changes walkthrough 📝

	Relevant files
Documentation	13 files with-tyk-multi-cloud.md Add Tyk Operator API Shard Tagging Example                              tyk-docs/content/advanced-configuration/manage-multiple-environments/with-tyk-multi-cloud.md Added a section on tagging an API for a shard using Tyk Operator. Provided a YAML example for API definition with tags. +23/-1    rate-limiting.md Add Tyk Operator Global Rate Limiting Example                        tyk-docs/content/basic-config-and-security/control-limit-traffic/rate-limiting.md Added an example of using Tyk Operator for global rate limiting. Provided a YAML configuration for setting rate limits. +22/-0    load-balancing.md Configure Load Balancing with Tyk Operator                              tyk-docs/content/planning-for-production/ensure-high-availability/load-balancing.md Introduced configuration for load balancing via Tyk Operator. Included a YAML example for enabling load balancing. +30/-1    analytics-plugins.md Add Tyk Operator Analytics Plugin Configuration                    tyk-docs/content/plugins/plugin-types/analytics-plugins.md Added Tyk Operator example for configuring analytics plugins. Provided YAML configuration for enabling analytics plugins. +49/-2    write-grpc-plugin.md Configure gRPC Plugin Hooks with Tyk Operator                        tyk-docs/content/plugins/supported-languages/rich-plugins/grpc/write-grpc-plugin.md Added examples for configuring gRPC plugin hooks using Tyk Operator. Included pre and post plugin hook examples. +53/-0    do-not-track-tyk-classic.md Enhance Middleware Configuration for Tyk Operator                tyk-docs/content/product-stack/tyk-gateway/middleware/do-not-track-tyk-classic.md Expanded section on configuring middleware in Tyk Operator. Added examples for disabling tracking for all or selected endpoints. +29/-4    request-header-tyk-classic.md Add Request Header Transformation Example for Tyk Operator tyk-docs/content/product-stack/tyk-gateway/middleware/request-header-tyk-classic.md Added example for API level request header transformations using Tyk Operator. Provided YAML configuration for adding and removing headers. +39/-2    url-rewrite-tyk-classic.md Reference Internal Looping Examples                                            tyk-docs/content/product-stack/tyk-gateway/middleware/url-rewrite-tyk-classic.md Added reference to internal looping examples. +2/-0      custom-plugins.md Document Custom Plugin Configuration with Tyk Operator      tyk-docs/content/product-stack/tyk-operator/advanced-configurations/custom-plugins.md Introduced documentation for configuring custom plugins using Tyk Operator. Provided a detailed example for JavaScript plugin configuration. +88/-0    internal-looping.md Document Internal Looping with Tyk Operator                            tyk-docs/content/product-stack/tyk-operator/advanced-configurations/internal-looping.md Added comprehensive documentation on internal looping with Tyk Operator. Included examples for URL rewrites, triggers, and proxy configurations. +344/-0  api-definition.md Update JavaScript Plugin Reference Link                                    tyk-docs/content/product-stack/tyk-operator/reference/api-definition.md Updated reference link for JavaScript custom plugins. +1/-1      cors.md Add CORS Configuration Example for Tyk Operator                    tyk-docs/content/tyk-apis/tyk-gateway-api/api-definition-objects/cors.md Added examples for enabling CORS in Tyk Operator. Provided YAML configuration for CORS settings. +43/-3    custom-analytics.md Setup Tag Headers with Tyk Operator                                            tyk-docs/content/tyk-apis/tyk-gateway-api/api-definition-objects/custom-analytics.md Added section on setting up tag headers with Tyk Operator. Included YAML example for tagging headers. +28/-0
Formatting	1 files response-header-tyk-classic.md Minor Formatting Adjustment                                                            tyk-docs/content/product-stack/tyk-gateway/middleware/response-header-tyk-classic.md Minor formatting adjustment. +1/-1
Configuration changes	1 files menu.yaml Update Menu with New Documentation Entries                              tyk-docs/data/menu.yaml Added menu entries for Custom Plugins and Internal Looping. +8/-0

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 496e273)

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Key issues to review Code Clarity The added YAML configuration example for tagging an API for a shard using Tyk Operator could benefit from a brief explanation or comment within the code to clarify the purpose and functionality of each section, especially for users who might not be familiar with YAML or the specific properties used. Documentation Consistency The example provided for setting a global rate limit using Tyk Operator should maintain consistency with the rest of the documentation. It's crucial to ensure that all examples follow the same format and structure to avoid confusion and maintain a high standard of clarity. Missing Explanation The section on configuring an analytics plugin with Tyk Operator lacks a detailed explanation or overview of what an analytics plugin does and why one might need to configure it. Adding a brief introduction could provide better context and enhance understanding for readers.

[netlify]

✅ PS. Pls add /docs/nightly to the end of url

Name	Link
🔨 Latest commit	d8fb658
🔍 Latest deploy log	https://app.netlify.com/sites/tyk-docs/deploys/66b2365173c65400080b63b4
😎 Deploy Preview	https://deploy-preview-5175--tyk-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[netlify]

✅ PS. Pls add /docs/nightly to the end of url

Name	Link
🔨 Latest commit	496e273
🔍 Latest deploy log	https://app.netlify.com/sites/tyk-docs/deploys/66c86a623332ac000851ea62
😎 Deploy Preview	https://deploy-preview-5175--tyk-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[letzya]

@dcs3spp Can I please ask why we have a new file for this capability and not updating an existing doc?
Also the page explains about looping using the operator while the word operator is not any of the titles. Why aren't we following the structure we had in other pages I saw (e.g. allowlist, virtual endpoint etc.)

[dcs3spp]

@letzya In spreadsheet provided by @caroltyk the target docs location was given as product-stack/tyk-operator/advanced-configurations/internal-looping.md within Tyk Operator docs

[letzya]

Please check if this is really needed or can be done in the same way as others.
If we still keep it this way (after we understand the reason), we'll need a link from the other looping page to this one

[dcs3spp]

@caroltyk what was the reason for targeting the internal looping example within Tyk Operators docs?

Are you ok with that content being moved to this page as an additional section?


[letzya]

@dcs3spp looks like this is a looping page that doesn't mention looping https://tyk.io/docs/product-stack/tyk-gateway/middleware/internal-endpoint-middleware/ - maybe we want to add the content here?

[caroltyk]

Hi @dcs3spp Since operator has special feature to allow user to reference rewrite URL using k8s name and namespace, we can keep this page where it is. We can add a link from https://deploy-preview-5175--tyk-docs.netlify.app/docs/nightly/product-stack/tyk-gateway/middleware/url-rewrite-tyk-classic/ to Operator's looping page as cross reference.

[dcs3spp]

@caroltyk I have updated with link from URL-Rewrite. Should there be a link updated from the Tyk Classic page of the section highlighted by @letzya https://tyk.io/docs/product-stack/tyk-gateway/middleware/internal-endpoint-tyk-classic/ ?

[caroltyk]

I don't think looping example should be added there because it doesn't do looping, just blocking external access.

Btw, we have Internal Endpoint example on another PR too: https://deploy-preview-5258--tyk-docs.netlify.app/docs/nightly/product-stack/tyk-gateway/middleware/internal-endpoint-tyk-classic/#tyk-operator

[caroltyk]

Suggest to create a subheading at the end of the page for Tyk Operator and add the link there, so that users easily know they can configure this via Operator from the navigation menu.

[dcs3spp]

Thanks @caroltyk I have updated to included subheading for Tyk Operator that includes the link

[dcs3spp]

Thanks @caroltyk I have added example for do not track middleware. Should we add an example to that section also for setting do not track middleware for a specific endpoint?

[caroltyk]

Thanks @caroltyk I have added example for do not track middleware. Should we add an example to that section also for setting do not track middleware for a specific endpoint?

Do not track middleware example is added here: https://deploy-preview-5258--tyk-docs.netlify.app/docs/nightly/product-stack/tyk-gateway/middleware/do-not-track-tyk-classic/#tyk-operator

[dcs3spp]

Thanks @caroltyk I have added example for do not track middleware. Should we add an example to that section also for setting do not track middleware for a specific endpoint?

Do not track middleware example is added here: https://deploy-preview-5258--tyk-docs.netlify.app/docs/nightly/product-stack/tyk-gateway/middleware/do-not-track-tyk-classic/#tyk-operator

Thanks @caroltyk I have added the example to https://deploy-preview-5175--tyk-docs.netlify.app/docs/nightly/product-stack/tyk-gateway/middleware/do-not-track-tyk-classic

[github-actions]

Persistent review updated to latest commit 496e273

[github-actions]

PR Code Suggestions ✨

Latest suggestions up to 496e273

Category	Suggestion	Score
Possible issue	Replace null values in CORS configuration with explicit methods and headers It's recommended to avoid setting allowed_methods and allowed_headers to null in the CORS configuration for the Tyk Operator API Definition. This can lead to unintended behavior where no methods or headers are allowed, which might not be the intended setup. Instead, specify the methods and headers explicitly or use defaults that allow typical web requests. tyk-docs/content/tyk-apis/tyk-gateway-api/api-definition-objects/cors.md [54-55] -allowed_methods: null -allowed_headers: null +allowed_methods: ["GET", "POST", "OPTIONS"] +allowed_headers: ["Content-Type", "X-Custom-Header"] Suggestion importance[1-10]: 8 Why: The suggestion addresses a potential issue where setting allowed_methods and allowed_headers to null could lead to unintended behavior. Providing explicit values improves clarity and ensures that typical web requests are allowed, enhancing the robustness of the configuration.	8
Enhancement	Provide actual values for placeholders in the JSON configuration example Replace the placeholder and in the JSON example with actual values to provide a complete and executable configuration example. tyk-docs/content/plugins/plugin-types/analytics-plugins.md [40-42] ```json { "analytics_plugin": { "enable": true, - "func_name": "<function name>", - "plugin_path": "<path>/analytics_plugin.so" + "func_name": "CustomAnalyticsFunction", + "plugin_path": "/usr/local/tyk-gateway/plugins/custom_analytics_plugin.so" } } <details><summary>Suggestion importance[1-10]: 7</summary> Why: Replacing placeholders with actual values enhances the example's usability and clarity, making it more practical for users to understand and implement. </details></details></td><td align=center>7 </td></tr><tr><td> <details><summary>Expand the list of tag headers in the example to include more common headers</summary> ___ **The example configuration for setting up tag headers with Tyk Operator should <br>include more common headers to demonstrate the feature's utility better. Adding <br>headers like <code>Referer</code> and <code>Authorization</code> can provide more practical examples of how <br>businesses can use tagging for analytics.** [tyk-docs/content/tyk-apis/tyk-gateway-api/api-definition-objects/custom-analytics.md [91-93]](https://github.com/TykTechnologies/tyk-docs/pull/5175/files#diff-23a35627af11d924b7a5b88273039d75890ee084e1ebd83f9196a2cb91f1ba0eR91-R93) ```diff tag_headers: - Host - User-Agent +- Referer +- Authorization Suggestion importance[1-10]: 6 Why: Adding more common headers like Referer and Authorization to the example configuration provides a more comprehensive demonstration of the feature's utility, making it more relevant for real-world applications.	6
	Performance	Increase the max_age value in CORS settings to reduce preflight requests The max_age field in the CORS settings for the Tyk Operator API Definition is set to 24, which is likely intended to be in seconds. However, this value might be too low for practical use, leading to frequent preflight requests. Consider increasing this value to improve client-side performance by reducing the number of preflight requests needed. tyk-docs/content/tyk-apis/tyk-gateway-api/api-definition-objects/cors.md [58] -max_age: 24 +max_age: 86400 # 24 hours in seconds Suggestion importance[1-10]: 7 Why: Increasing the max_age value from 24 seconds to 86400 seconds (24 hours) is a practical improvement that reduces the frequency of preflight requests, enhancing client-side performance without compromising security.	7
Maintainability	Capitalize the first letter of each word in menu titles for consistency Ensure consistency in the menu titles by capitalizing the first letter of each word in the new menu items for better readability and alignment with existing menu items. tyk-docs/data/menu.yaml [3884-3888] - title: "Custom Plugins" -- title: "Looping" +- title: "Internal Looping" Suggestion importance[1-10]: 5 Why: Ensuring consistency in menu titles by capitalizing the first letter of each word improves readability and aligns with the existing style, contributing to better maintainability of the documentation.	5
	Correct the indentation of the tags field in the YAML example Ensure that the tags field in the YAML example is properly indented under the spec section to maintain the correct YAML structure and hierarchy. tyk-docs/content/advanced-configuration/manage-multiple-environments/with-tyk-multi-cloud.md [58-59] ```yaml apiVersion: tyk.tyk.io/v1alpha1 kind: ApiDefinition metadata: name: httpbin spec: name: httpbin use_keyless: true + protocol: http + active: true tags: - edge - protocol: http - active: true proxy: target_url: http://httpbin.org listen_path: /httpbin strip_listen_path: true <details><summary>Suggestion importance[1-10]: 3</summary> Why: The suggestion addresses a minor indentation issue that does not affect functionality but improves readability and maintainability of the YAML structure. </details></details></td><td align=center>3 </td></tr></tr></tbody></table> ___ #### Previous suggestions <details><summary>Suggestions up to commit d8fb658</summary> <br><table><thead><tr><td>Category</td><td align=left>Suggestion&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </td><td align=center>Score</td></tr><tbody><tr><td rowspan=1><strong>Compatibility</strong></td> <td> <details><summary>Include the <code>response_processors</code> segment for compatibility with older Gateway versions</summary> ___ **To ensure the <code>ApiDefinition</code> is correctly configured for all versions, include the <br><code>response_processors</code> segment within the <code>spec</code> for compatibility with Gateway versions <br>prior to v5.3. This ensures that the header modifications are processed correctly <br>across different versions.** [tyk-docs/content/product-stack/tyk-gateway/middleware/request-header-tyk-classic.md [149-182]](https://github.com/TykTechnologies/tyk-docs/pull/5175/files#diff-2b8d7eb6abcd81e52f5955ba294009309b94ab4db7a34d53274266d94340b985R149-R182) ```diff ```yaml apiVersion: tyk.tyk.io/v1alpha1 kind: ApiDefinition metadata: name: httpbin-global-headers spec: name: httpbin-global-headers use_keyless: true protocol: http active: true proxy: target_url: http://httpbin.org listen_path: /httpbin-global-headers strip_listen_path: true + response_processors: + - name: header_injector version_data: default_version: Default not_versioned: true versions: Default: name: Default use_extended_paths: true paths: black_list: [] ignored: [] white_list: [] global_headers: foo-req: my-foo bar-req: my-bar global_headers_remove: - hello <details><summary>Suggestion importance[1-10]: 8</summary> Why: The suggestion ensures compatibility with older versions of the Tyk Gateway, which is important for users who may not be on the latest version. This addition helps maintain functionality across different versions, addressing a potential issue for some users. </details></details></td><td align=center>8 </td></tr></tr></tbody></table> </details>