Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] How to not include "Managed" gateway by default when publishing APIs #501

Open
daviian opened this issue Mar 12, 2024 · 13 comments · May be fixed by #515
Open

[Question] How to not include "Managed" gateway by default when publishing APIs #501

daviian opened this issue Mar 12, 2024 · 13 comments · May be fixed by #515
Labels
enhancement New feature or request question Further information is requested

Comments

@daviian
Copy link
Contributor

daviian commented Mar 12, 2024

Release version

v5.1.1

Question Details

Our APIM solely uses self-hosted gateways and thus does not require every API to be assigned to the "Managed" gateway.
However, even though the API is only assigned to a specific self-hosted gateway, the "Managed" gateway is still getting assigned to it.

I added a "Managed" gateway into the gateways directory and set the APIs to an empty array.
However, this only works when not using the commit id during publishing, as the Managed gateway file is not changed when I add a new API.

How is this supposed to be handled?

Expected behavior

"Managed" gateway is not assigned whenever I assign it to some custom gateways.

Actual behavior

"Managed" gateway is always assigned to new APIs.

Reproduction Steps

Create a fresh API and assign it to a custom gateway (not the "Managed" one)

Folder structure:
image

Content of gateways/self-hosted/apis.json

[
  {
    "name": "my-new-api"
  }
]

Content of gateways/Managed/apis.json

[]

Since the gateways/Managed/apis.json has not been touched/changed during the commit, the publisher does not update the Managed gateway assignment, so the new API is actually having both the "Managed" and "self-hosted" gateways assigned to it.

Copy link

  Thank you for opening this issue! Please be patient while we will look into it and get back to you as this is an open source project. In the meantime make sure you take a look at the [closed issues](https://github.com/Azure/apiops/issues?q=is%3Aissue+is%3Aclosed) in case your question has already been answered. Don't forget to provide any additional information if needed (e.g. scrubbed logs, detailed feature requests,etc.).
  Whenever it's feasible, please don't hesitate to send a Pull Request (PR) our way. We'd greatly appreciate it, and we'll gladly assess and incorporate your changes.

@waelkdouh waelkdouh added the question Further information is requested label Mar 12, 2024
@guythetechie
Copy link
Contributor

There is a similar issue with products in #245. This new implementation will fix it, and we're adding a similar one for gateways. Should be in the next major release.

@guythetechie guythetechie added the enhancement New feature or request label Mar 12, 2024
@guythetechie guythetechie moved this to 🏗 In progress in APIOPS Roadmap Mar 12, 2024
@daviian
Copy link
Contributor Author

daviian commented Mar 12, 2024

I have my doubts that the proposed changes actually resolve this issue.
Let me share my train of thought and feel free to correct me if I am wrong.
Every addition of an API contains a changed file in the self-hosted gateway folder, so this is all good. The publisher considers them during publishing of the git commit.
However, since I don't want to have the new API associated with the existing managed gateway, there are no file changes in the managed gateway folder, so the publisher is still not going to consider them.
How is the proposed change addressing this?

@guythetechie
Copy link
Contributor

I'm not sure what your baseline is, so I'll make some assumptions and walk through a few scenarios.

Let's say you start with a managed and self-hosted gateway in the portal. You have zero APIs. You run the extractor. Your folder structure will be:

  • artifacts
    • gateways
      • managed
        • gatewayInformation.json
      • self-hosted
        • gatewayInformation.json

You then go to the Azure portal, create APIs A and B, assign API A to the managed gateway, and API B to the self-hosted gateway. You run the extractor. Your folder structure will be:

  • artifacts
    • gateways
      • managed
        • gatewayInformation.json
        • apis
          • apiA
            • gatewayApiInformation.json
      • self-hosted
        • gatewayInformation.json
        • apis
          • apiB
            • gatewayApiInformation.json
    • apis
      • apiA
        • apiInformation.json
      • apiB
        • apiInformation.json

You then decide to do a code-first deployment: instead of making changes via the portal & extracting, you want to change the git artifacts and publish that commit. In this change, you remove API A from the managed gateway, add a new API C to the self-hosted gateway, and add a new API D that won't be associated with any gateways. Your folder should look like this:

  • artifacts
    • gateways
      • managed
        • gatewayInformation.json
      • self-hosted
        • gatewayInformation.json
        • apis
          • apiB
            • gatewayApiInformation.json
          • apiC
            • gatewayApiInformation.json
    • apis
      • apiA
        • apiInformation.json
      • apiB
        • apiInformation.json
      • apiC
        • apiInformation.json
      • apiD
        • apiInformation.json

The publisher will detect the following changes and make them accordingly when it runs:

  • artifacts/apis/apiC/apiInformation.json was created. API C will be created in APIM.
  • artifacts/apis/apiD/apiInformation.json was created. API D will be created in APIM.
  • artifacts/gateways/managed/apis/apiA/gatewayApiInformation.json was deleted. API A will be removed from the managed gateway.
  • artifacts/gateways/self-hosted/apis/apiC/gatewayApiInformation.json was created. API C will be added to the self-hosted gateway.

Does that make sense? Let me know if I'm missing something.

@guythetechie
Copy link
Contributor

Something just clicked for me; I misread your initial question. When you talk about the "managed" gateway, you're not referring to a gateway you create called "managed". You're referring to the built-in managed gateway.

If that's the case, I doubt there's functionality to manage that "managed" gateway. You can't export it as an artifact and "manage" it. You can assign APIs to a self-hosted gateway using the methods described in the previous comment, but I'm not aware of a way to also disassociate them explicitly from the built-in gateway.

I would suggest creating a support ticket and asking whether that's possible (outside of our tools). If they show you how to do it, we can see if the steps are achievable through our tools.

@daviian
Copy link
Contributor Author

daviian commented Mar 12, 2024

Yes, I am talking about the built-in managed gateway. Sorry for the missing clarity.

I actually experimented and simply created a folder called "Managed" and added an apis.json that has an empty array in it.
When I use the publisher without the COMMIT_ID it actually disassociates the APIs from the built-in gateway.
But as I said, this does not work when only looking at changes within a git commit.

@guythetechie
Copy link
Contributor

Can you elaborate on what you're seeing? How do you know they're no longer associated with the built-in gateway?

@daviian
Copy link
Contributor Author

daviian commented Mar 12, 2024

You can see the assigned built-in gateway when you have a look into the API in the portal.
The field "Gateways" either contains the "Managed" + your custom ones, or only the custom ones after I tested it.

image

@guythetechie
Copy link
Contributor

Thanks for the information - makes sense now. I think our implementation will partially handle your scenario.

When the extractor runs, it queries APIM for a list of gateways. It then loops through all gateways and exports their associated APIs. The problem is that APIM's list of gateways does not include the managed gateway. We would have to add a step to explicitly ask for the managed gateway's APIs. The APIs would then show up in artifacts/gateways/managed/apis/....

Adding to our backlog for further investigation.

@guythetechie guythetechie moved this from 🏗 In progress to 📋 Backlog in APIOPS Roadmap Mar 12, 2024
@waelkdouh
Copy link
Contributor

@daviian feel to submit a PR and we can merge accordingly.

@daviian
Copy link
Contributor Author

daviian commented Mar 13, 2024

Thanks for the information - makes sense now. I think our implementation will partially handle your scenario.

When the extractor runs, it queries APIM for a list of gateways. It then loops through all gateways and exports their associated APIs. The problem is that APIM's list of gateways does not include the managed gateway. We would have to add a step to explicitly ask for the managed gateway's APIs. The APIs would then show up in artifacts/gateways/managed/apis/....

Adding to our backlog for further investigation.

I suppose this also needs to be done in the publisher? Otherwise the built-in managed gateway is always assigned automatically.

@daviian
Copy link
Contributor Author

daviian commented Mar 29, 2024

@guythetechie - I've had a look into it and locally it looks good so far. However, there is one point I have concerns about.
The APIM API documentation Gateway Api - Create Or Update and Gateway Api - List By Service excludes "managed" from being an allowed value ("Must not have value 'managed'").
However, both APIs work as intended for our use case when passing "managed". Additionally this API is also used from the APIM Portal when assigning/removing APIs from the managed gateway, so I assume the API documentation requires an update.
The only API that returns a validation error is Gateway - Get, so I conditionally exclude fetching the gateway details for the "managed" gateway.

What are your thoughts on this? If you like I can create the PR so that you can have a preliminary look as well.

EDIT:
There has been a similar question https://learn.microsoft.com/en-us/answers/questions/624766/azure-apim-gateways-and-bicep that has not been answered since quite a while.

@daviian daviian linked a pull request Mar 29, 2024 that will close this issue
@daviian
Copy link
Contributor Author

daviian commented Apr 15, 2024

@guythetechie @waelkdouh Any update? Anything I can do/provide to help out and create some progress?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
Status: 📋 Backlog
Development

Successfully merging a pull request may close this issue.

3 participants