Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix service registry clustering #4

Open
wants to merge 16 commits into
base: master
Choose a base branch
from

Conversation

ess
Copy link
Contributor

@ess ess commented Nov 4, 2022

There were previously two issues with registry server:

  • The count configuration wasn't being honored. This was due to a dangling bit of logic in the provision/update workflows around scaling the service-registry app. This has been rectified.
  • For some reason, if we get an app object from capi via ccv3, we can't actually pass that same object because capi raises an error regarding the relationships field that is part of that object. I've worked around this in update_registry_server_instance by creating a temp copy of the app object without relationships and passing that instead.

Dennis Walters added 3 commits November 3, 2022 18:30
RegistryParams is no longer a raw map[string]interface{}

Instead, it is a struct with raw values that are json-capable.
This was failing silently in previous builds. In the release
prior to this change, it was failing *loudly*.

Now it's no longer failing.
For some reason, the somewhat aged ccv3 client that is
in our vendor bundle is unable to `UpdateApplication`
if given an application object retrieved from the API.

The error presented is "Unknown field(s): relationships"
and is being generated on the capi side of the equation.

We've worked around this by creating a temporary copy of
the app object without its relationships collection and
passing THAT temp object to the update call.

It appears that this likely also affects update_config_server
workflow, and we should double-check that.
@ess ess requested a review from TheDigitalEagle November 4, 2022 08:01
@ess ess self-assigned this Nov 4, 2022
@ess
Copy link
Contributor Author

ess commented Nov 4, 2022

@TheDigitalEagle Before merging this, please try updating a running config-server instance and let me know if you'd like me to apply my fix to the update_config_server_instance workflow.

Dennis Walters and others added 13 commits November 4, 2022 08:09
`utilities.SafeApp(ccv3.Application) ccv3.Application`
This is to avoid doing partial service-registry creations/updates
in the event that we receive an invalid (<1) desired node count.
We confirmed that the config server update workflow is
affected by the same issue that utterly plagued the
registry server update workflow.

So we put in the same fix.
In an attempt to get registry server peering working correclty,
we're now presenting the internal connection info for each
registry server process instance as a peer.

In a perfect world, we'd be doing per-process-instance peer
configuration, but we've yet to find a way to do this.
This appears to be the only viable way to provide information
to the process instances that can be used to derive a working
service registry peering configuration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant