KBS: Optimize performance and memory usage

[Xynnn007]

Currently we are using a global Mutex to protect the only one attestation service client from unsafe thread sync & send. This brings performance bottle neck.

This commit will create a temporary client object to handle the current attestation request, which can break the mutex.

Fixes #256

[fitzthum]

I think this is good but see one comment

[fitzthum]

Any concerns about not locking here given that this will change the state of the AS?

[Xynnn007]

Thanks for pointing out this. IMO, the lock/synchronization mechanism should be token cared of on the server side, like in CoCo-AS we use a read-write lock to handle the requests.

The client side code just call the API and the server can response in some order. That means, if users want to use a new policy to do attestation. In their code should call set_policy of KBS and wait for the successful return. Then call attestation API.

Among current three implementations of AS client, built-in-coco-as seems to have some trouble for synchronization. This is because AS is just a lib and KBS is actually the server. Let me add aRwlock on the client side as a workaround.

[Xynnn007]

We are doing some tests together with @Lu-Biao to check if this works.

[Xynnn007]

With much help from @Lu-Biao , we found some bottle necks:

Throughput

1. KBS uses session to save the context of an RCAR handshake protocol. This session is currently saved globally using a Mutex-modified HashMap, which has become the current concurrency performance bottleneck.
2. KBS uses a single gRPC client globally to connect to gRPC-CoCo-AS, and all requests will be processed serially.
3. A Mutex is used for the AS object during KBS programming. In fact, it is only mutually exclusive when modifying the policy operation.

Memory usage

1. Currently, the global Hashmap that manages sessions only records new entries. Old expired session records are actually useless, but they will not be eliminated, resulting in a one-way increase in memory usage.

We did some optimizations

1. Replace the HashMap that manages sessions with a concurrently safe HashMap, abandon Mutex, and improve concurrent throughput.
2. Create a new gRPC client connection pool mechanism. On the one hand, it ensures that attestation requests can be processed concurrently and connections are reused. On the other hand, it prevents too many connections from being created at the same time and exhausting the temporary ports at the bottom of the operating system.
3. Encapsulate concurrency security inside the Attestation object and change Mutex to read-write lock RwLock
4. Add a coroutine (green thread in Tokio) to regularly release expired sessions in the session map

[Xynnn007]

The following are the test results before and after optimization

Throughput without the patch

Throughput with the patch

Throughput increased by ~5x

Memory usage will never increase endlessly.

Memory usage without the patch ( ~7min)

Memory usage with the patch ( ~1hour)

[fitzthum]

This is great. I really like the new SessionStatus enum (even though the variants share some of the same fields) and the macro for its fields. Ofc the results are great too.

I have one minor comment but generally seems ready.