Infinite loops and other challenges with arbitrary policies

[MartenvanWezel]

I'm aware OPA has certain provisions/tips - https://www.openpolicyagent.org/docs/latest/policy-performance/#high-performance-policy-decisions - to ensure (is it.. sure?) that evaluations don't take too long.

But I am wondering if it is feasible to deploy OPA in such a way that it should "never" leave the system in an unwanted state, e.g. in some infinite loop, or demanding unreasonable amounts of memory (without building monitoring to detect it externally and killing misbehaving threads).

Of course main reason for this question is the worry that opening up the pipeline for unvetted users can cause an accidental or maliciously formed query to bring critical systems to its knees.

And of course ideally the answer should keep the power of the rego language intact :)

[srenatus]

There's a few aspects here, and since I'm not entirely certain about your scenario (rego from untrusted users?), I'll present them in brief -- let me know if you need more detail on any of these:

1. Rego doesn't allow for recursion -- at all. If you find a way to create a policy that would somehow get stuck in an infinite loop, that's a bug, please file it.
2. You can use capabilities to selectively disable certain builtins -- like http.send -- that could do stuff you don't want the policies to do.
3. Limiting memory and CPU resource use is tricky with the Rego interpreter that's used inside of OPA. One way forward here would be to use the Wasm modules that OPA can build for your policies, and configure the wasm host engine accordingly. The notion of a "gas" level that can "run out" seems to be available in several engines, and could help here.
4. As a stop-gap for limiting unbounded resource use in policy evaluations, timeouts are enforced. If the requesting HTTP client goes away, the evaluation will be cancelled. If you've got some sort of proxy in front of OPA (nginx etc) you could give each request a certain amount of max time to use, and if it's can't be resolved in that time frame, the caller will know -- but more importantly, perhaps, the resources will no longer be hogged. (⚠️ This can't be guaranteed for custom or third-party built-in functions, but it should hold true for all that's included in OPA.)

I'm curious if this helps you at all, and if you need more pointers on any of those items...

[MartenvanWezel]

Rego for untrusted users> well, I suppose this is true for any codebase but just observing the fact that not every piece of code that gets checked in to a repo and gets pushed to production works well. If that code happens to be policy in a centralised policy engine then I can imagine 'bad things' might happen.

1> Recursion> ah good to know. Given that we can still iterate and then iterate on subresults I imagine you can still tie up a CPU fairly easily.

2> Yes was aware to disable http.send but I imagine there's more stuff that's somewhat worrying intermixed with useful stuff if I try to disable builtins at that level

3> OK, it's worth exploring. I guess not many languages have protections at this level.

4> Perhaps also some type of staging pipeline could be used to test performance of some queries. Would that be a sane approach?

[srenatus]

Perhaps also some type of staging pipeline could be used to test performance of some queries.

Yeah definitely. Nothing prevents you from writing policies that take long to evaluate, even more so given large data sets. There are some hints about what to avoid in the performance docs but nothing beats measuring this yourself, and at best, you don't do that in a production environment.