Permanently turn-off Security Manager (JSM) starting 3.0 #17181
Comments
kumargu
added
enhancement
|
Thanks @kumargu for putting together the proposal. Can we clearly state capabilities addressed and unaddressed by alternatives, preferably in table format? This will be helpful for community to provide feedback. |
|
FYI @sam-herman, I know you were supportive of removing JSM in the past, would love to have your thoughts captured here as well. |
|
There is another option; compatible migration with backward compatibility. Keep the API's in place, but don't enable SM on platforms that don't support it. Any decision to remove SM can be left until OpenJDK removes the API's that have been left in place for binary compatibility (no-op's). Don't use -Djava.security.manager=, instead try System::setSecurityManager catch UnsupportedOperationException. Update all permission checks to use Guard::checkGuard, if SM is installed it will retrieve it and check the permission, otherwise it's a no op. These changes are compatible across all versions. I remember a time when people thought Java serialization was secure, I had arguments on development lists and I couldn't get developers to believe me it wasn't secure. Log4J was relatively recent, I don't think this one's over yet. |
|
i think i missed tagging dB @dblock |
|
Listing the extract of the permissions from the OpenSearch core: The goal at the moment is to cover only these two (by POCing Java agent instrumentation): |
Thanks @pfirmstone , I believe it is possible, we could do per-JDK tuning and there shouldn't be any blockers to support pre/post JDK-24s within same codebase. However, there is a significant long time cost in doing so that we have been debating recently (brought by @andrross @kumargu): we know that JSM is out for good, and since 3.0 is the release line we foresee being supported for at least the next few years, cutting it off now would pay off in a medium/long term. |
To be clear, the significant long term cost is in having to make all future code play nice with the security manager enabled. Virtual threads is one example today where that is difficult. I suspect this will only get harder as the JDK evolves without JSM present. |
High level comparison of SM vs. systemd Security Configurations(the comparison excludes File and Socket permissions, because they would be covered just like today via the java agent)
|
@krishna-ggk: updated |
|
Thank you @kumargu for putting this together and creating a comparison table of SM vs systemd configuration. I'm supportive of this RFC, given the direction that Java is taking. There's been a lot of good discussion about what JSM currently provides OpenSearch and limitations around the various replacements (or combinations thereof). JSM has given OpenSearch a very flexible security model that sandboxes the plugins to the point where plugins must "opt into" privileged operations and cluster administrators must accept that on installation. Unfortunately, that flexible security model has also made JSM historically difficult to configure and comprehend and resulted in lower adoption of JSM overall. With any replacement (even if it can't fully replace JSM), I'd like to retain the opt in model and prompt cluster administrators on installation - perhaps with more human-readable language on the prompt than the current dump of the policy file. I've been reading up on Java Permissions more closely to try to categorize JSM protections at a high level. As discussed in both RFCs, FileSystem and Network access are 2 important pillars, but I've captured a few others as well:
JSM certainly limits the blast radius for any zero-day RCE exploit. For an exercise, I wrote a custom endpoint in a plugin that accepts code as a param and executes the code. With any replacement we introduce for JSM, I'd love to have testability where we can enumerate through different scenarios and automate the checks. |
I really like the idea of removing Security Manager in favor of delegating access control to the Operating System.
In my opinion, whatever access control mechanism for users/process/threads on the Operating System will be the easiest since it allows for the user to accommodate to their environment. |
+1 on emphasising on this point. |
Sorry, I am not following the opt-in model you mentioned here. In general, if we think we are convinced with the new security postures, we should drop the older one, otherwise, we and cluster administrators would be in a constant loop of evaluation. After the dawn of security manager, I would assume nobody talking about JSM and we'd rather focus on how we can improve the then security posture. |
Another point I forgot to mention is around native dependencies of OpenSearch/plugins. For example KNN plugin is using native dependencies which are outside of the JVM cage. Therefore security manager will not be useful in any way with those. |
Is your feature request related to a problem? Please describe
This is a proposal to gather community feedback on the permanent deprecation of the Security Manager (JSM) starting in OpenSearch 3.0
The Security Manager will reach its end-of-life in JDK 24 (see https://openjdk.org/jeps/486 #1687). OpenSearch 3.0, which will be bundled with JDK 21, could potentially still support the Security Manager. However, we propose to disable it starting in version 3.0 (see [1] for reasoning). By "disabling," we mean that OpenSearch will not initialize the Security Manager during the bootstrap process. If we decide to turn it off in 3.0, there will be no option to revert this decision—meaning the Security Manager will be permanently disabled in OpenSearch.
[1] JDK 21 introduces virtual threads, based on: JEP 444: Virtual Threads, "Virtual threads have no permissions when running with a SecurityManager set." This means that any code using virtual threads under a Security Manager will not work. While OpenSearch 3.0 itself will not contain virtual threads, we anticipate that developers—especially plugin developers—will want to leverage them in future 3.x versions. To support this, we could consider disabling the Security Manager in a later 3.x release. There is a strong argument for not deprecating a security feature unless absolutely necessary. However, deprecating such a significant security feature in a minor version release is generally not acceptable. An alternative approach could be to introduce a new major version (4.0) with the Security Manager disabled. However, it is too early to predict the lifecycle of major versions, as they tend to last a long time (for example, the 2.x series lasted around two years).
Over last few months we have been working on finding alternatives (also discussed later) for security manager. The alternatives are not a drop-in-replacement of security manager. Our research into alternatives has been extensive, and we believe the same solutions could be applicable across OpenSearch 3.0, 3.x, and 4.x. Hence we propose disabling in the major version release of 3.0
The most critical question remains: "Are these alternatives sufficient?" That is the feedback we seek from the community. If there are no major blockers, we will proceed with disabling the Security Manager in 3.0. Otherwise, we will delay the decision until absolutely necessary or until we find a better replacement.
Describe the solution you'd like
We propose below two alternatives as replacement for security manager.
[Security Manager Replacement] Strengthen OS core security via systemd configuration #16729:
Systemd offers a whole lot of security configs to isolate linux processes as well as isolate a bad actor effecting the underlying operating system. The core idea is to protect the Opensearch process from being effected (e.g system exit) from untrusted code (certain plugins). It can allow or blocks specific system calls using seccomp filters heavily reducing the attack surface of the exposed kernel. It can be used to allow or deny access to socket and files.
Java Agent [POC] [Security Manager Replacement] Native Java Agent (dynamic code rewriting, must be low overhead) #16731: With security manager we can define permissions at plugin level, elevating socket, file access or thread context permissions at an individual plugin level. This is not possible with
systemdbecause it rules are defined at process level. Elevating permissions at process level is unacceptable. To mimic security manager: we introduced a "Java agent" which is a special type of class (in Java) that can modify the bytecode at runtime. They use Java Instrumentation API to intercept and modify the behavior of applications running on the JVM. We can launch a java agent which can intercept calls like SocketChannel.connect() or File.createNewFile() enforcing a security-policy which decides if a FileIO or Socket connect is allowed or not. Interestingly we can apply the same security policies that are being used today for plugins (a less of a breaking change). However, writing an interceptor requires a whole lot of boiler-plate code and hard for maintainability so we take the trade-off to implement interceptors only for Socket and File access (the most dangerous ones)."While these are the two alternatives we are initially proposing and could be available in 3.0, we remain open to other solutions that can be incorporated as we continue enhancing security in layers."
No going back:
Once we have turned-off security manager we actually allow developers to run their code with no more wrapping around
AccessController.doPrivileged()block. An attempt to turn-back-on the security manager means breaking these code paths, hence no going back.Late clean-up of
AccessController.doPrivileged():Executing code within AccessController.doPrivileged() with the Security Manager disabled neither enforces any policy evaluation nor has any side effects—it effectively becomes dead code. This dead code will not be removed in 3.0 but may be cleaned up in later versions as a non-breaking change.
Comparison of Java Security Manager vs. systemd Security Configurations
(the comparison excludes File and Socket permissions, because they would be covered just like today via the java agent)
java.lang.management.ManagementPermission "control"java.lang.management.ManagementPermission "monitor"java.lang.reflect.ReflectPermission "suppressAccessChecks"NoNewPrivileges=true(Prevents privilege escalation)java.lang.RuntimePermission "accessClassInPackage.*"MemoryDenyWriteExecute=true(Prevents modification)java.lang.RuntimePermission "createClassLoader"NoNewPrivileges=true(Prevents spawning new privileged processes)java.lang.RuntimePermission "setContextClassLoader"java.lang.RuntimePermission "modifyThread"RestrictNamespaces=true(Prevents manipulation)java.lang.RuntimePermission "modifyThreadGroup"RestrictNamespaces=true(Prevents manipulation)java.lang.RuntimePermission "shutdownHooks"CapabilityBoundingSet=~CAP_SYS_ADMIN(Restricts shutdown abilities)java.net.NetPermission "getProxySelector"RestrictAddressFamilies=~AF_INET ~AF_INET6(Blocks internet)java.net.NetPermission "setDefaultAuthenticator"java.security.SecurityPermission "createAccessControlContext"ProtectSystem=full(Limits modifications to system-wide security settings)java.security.SecurityPermission "insertProvider.SaslPlainServer"ProtectSystem=strict(Prevents unprivileged modifications)java.util.logging.LoggingPermission "control"java.util.PropertyPermission "os.name", "read"ProtectSystem=full(Restricts system property tampering)java.util.PropertyPermission "opensearch.path.conf", "read,write"ReadWritePaths=/var/lib/opensearch(Controls file-level permissions)java.util.PropertyPermission "opensearch.allow_insecure_settings", "read,write"ProtectSystem=strictjavax.management.MBeanServerPermission "createMBeanServer"javax.management.MBeanTrustPermission "register"javax.security.auth.AuthPermission "doAs"NoNewPrivileges=true(Restricts privilege escalation)javax.security.auth.AuthPermission "modifyPrincipals"javax.security.auth.AuthPermission "modifyPrivateCredentials"MemoryDenyWriteExecute=true(Prevents execution in writable memory)RestrictRealtime=true(Blocks real-time scheduling)LockPersonality=true(Prevents ABI changes)ProtectClock=true(Prevents time manipulation)ProtectKernelModules=true(Prevents module loading)ProtectKernelLogs=true(Prevents reading logs)ProtectControlGroups=true(Prevents cgroup modifications)RestrictNamespaces=true(Prevents unprivileged namespace changes)RestrictSUIDSGID=true(Prevents privilege escalation via SUID binaries)So, let us know your thoughts!
The text was updated successfully, but these errors were encountered: