Encrypt keys before saving in OMAP file

[gbregman]

Fixes #960

[epuertat]

@gbregman if this is about encrypting secrets at rest (in OMAP values), why the client is encrypting them? With encryption at rest, everything happens close to the storage layer (librados omap read and write calls in the server), but not end to end (since gRPC already provides in transit encryption through TLS).

By having both parties (client & server) to share the encryption key partially defeats the purpose of encryption and also complicates deployment. The secure storage of secrets should be a responsibility of the server, not the client side (leaking a secret from a single client side exposes all secrets).

[baum]

Could you share your thoughts on this draft for nvmeof gateway KMS integration: https://github.com/baum/ceph-nvmeof/blob/kms/KMSclient.md ? The KMS client feature appears to be a recurring pattern in ODF, seen in components like noobaa-operator, rook, RGW, etc. I would consider this approach for nvmeof gateway.

[baum]

Might be an interesting read:

• Configuring the Ceph Object Gateway to use SSE-KMS with Vault
• Key Management System/Rook
• Ceph Object Gateway Config Reference

[baum]

Should we consider a generic KMS support layer aligned with other Ceph components?

We already said several times that this wouldn't be done for now. It might be done in the future.