OPA dynamic cluster without bundles

[Tristan-GG]

Hello,

I want to deploy an OPA for my application because I need to assign rights to users of my application directly through it.
The problem is that the production application needs to change rights very dynamically, and with our infrastructure restrictions, bundles are not an option.
What's more, we need a robust OPA, so we opt for a cluster.

But how do you synchronize data with an OPA in cluster mode? At the moment I'm on a test environment with docker (but later for the prod with Ansible) where I replicate my service 3 times to avoid any loss of data and so that the OPA is in high availability, but I can't find a solution to synchronize the dynamically recorded data rules. (the rules are directly set up by postman on OPA, and I want to keep this function preferably)

Is it possible to do without bundles in my case? Is there a solution to obtain the same highly dynamic operation but with bundles? or by storing the data on a shared volume or file, for example?

Thanks in advance for your reply!

[anderseknert]

Hi there!

It would help to know what those infrastructure restrictions are. OPA can't reach out to a remote host?
I suppose you could be running a bundle server on the same host then? Can be a tiny nginx or something, not exposed outside of the host.

[Tristan-GG]

I've just had a meeting with the development team, as we had a number of points to clarify following the day's brainstorming sessions.

we're going to use bundles because I don't think it's possible to manage everything dynamically while maintaining data persistence and robustnes

I do have one question !

If we have an OPA cluster with 3 nodes. Our OPA service is taking up more and more data, and is generating bundles at a specific time. Some time later, one of the nodes falls, so we have to reload the bundles, except that this node is now behind the others, which have taken dynamic writes, but not yet been bundled. Do you have a solution for this, even if I already have my own idea?

[anderseknert]

The OPA instance getting restarted would pull down the latest bundle from the same endpoint as the other instances, and would not be ready until it had the bundle fully loaded.

which have taken dynamic writes,

I'm not sure what this means. Can you elaborate? 🙂

[Tristan-GG]

I was referring to the fact that any changes made to OPA that have not yet been bundled will be lost if an opa crashes, as the bundle it will be reloading will not be up to date with the changes made in the meantime. Does OPA have a way of resynchronizing a node that has retrieved the latest bundle without the latest modifications?

I'm also starting to think about adding OPAL to OPA or CONSUL.
OPAL in direct operation or via git to add backup but also instant modifications on my various OPA nodes as well as a data backup.
CONSUL for bundle file sharing, where OPA loads the shared bundle to update the rules.

I'm beginning to think, and I may be wrong, that OPA on its own isn't ready for a standalone prod because it lacks robustness due to its standalone operation.

If you ever have any advice on how to set up a prod or on adding OPAL, CONSUL or other !

[anderseknert]

OPA is a distributed component, and not a single source of truth for data. And like any distributed component, there are no guarantees as for things like data synchronization, but eventual consistency. OPA should never carry data that isn't readily available elsewhere!

For scenarios where you need to ensure as low latency as possible, minimize inconsistent data and ensure offline restarts work:

1. Enable long polling to have OPA fetch bundles immediately when updates are available.
2. Set persist: true on bundles in configuration to have persistence enabled across restarts.
3. (Optionally/possibly) delta bundles.

[Tristan-GG]

Ok, thank you very much for your answers, I think I've got a good idea of where I'm going to start. I'm closing the subject because the question of setting up opa without bundles is no longer topical, as you can't achieve a prod environment with data persistence without setting up bundles.

I am grateful for your answers!