Performance Testing

This summarizes the very first performance testing done with MSAL Go. Below is a description of test environments and findings.

Test set up

Method used for testing is AcquireTokenForClient() and AcquireTokenSilent(). An external storage was used which was partitioned based on the suggested Cache key given out by MSAL Go. The external storage used is this https://github.com/patrickmn/go-cache This scenario is relevant for multi tenant applications. The cache key is of the format "clientid_tenant_id_AppTokenCache"

The input scenario is there are n number of tenants and t is the total number of tokens. So, as an example, 8 tenants will create 8 partitions and t number of tokens are distributed evenly in these partitions. It was tested on a SurfaceBook2(Intel(R)Core(TM) i7-8650U CPU @ 1.90GHz, 16GB RAM, 4 cores) and a Windows 10 Enterprise

Results

The results below suggest that using the recommended cache partitioning key, it shows constant time lookup regardless of how many items are in the cache. Ideally the number of tokens per cache partition would be less than 10 tokens but the results are comparable even when the number of tokens per cache partition goes upto a 100.

Action	Function	# of Tokens	# of Tenants	Tokens per cache partition	Avg Time	P50	P95
500
Populate Cache	AcqTokenClient	500	10	50	1.091997ms	999.9µs	2.9999ms
Retrieve from Cache	AcqTokenSilent	500	10	50	1.687641ms	1.9912ms	2.9994ms
1,000
Populate Cache	AcqTokenClient	1000	10	100	1.705192ms	1.9984ms	3.0458ms
Retrieve from Cache	AcqTokenSilent	1000	10	100	3.883097ms	3.9985ms	5.0484ms
5,000
Populate Cache	AcqTokenClient	5000	100	50	948.985µs	999.8µs	2.0287ms
Retrieve from Cache	AcqTokenSilent	5000	100	50	1.994151ms	1.9999ms	3.0019ms
50,000
Populate Cache	AcqTokenClient	50,000	1000	50	896.284µs	999.8µs	2.0011ms
Retrieve from Cache	AcqTokenSilent	50,000	1000	50	2.209502ms	2.0001ms	3.0044ms
200,000
Populate Cache	AcqTokenClient	200,000	10,000	20	716.974µs	996.8µs	2.0011ms
Retrieve from Cache	AcqTokenSilent	200,000	10,000	20	1.304306ms	1.0014ms	2.6376ms
100,000
Populate Cache	AcqTokenClient	100,000	1000	100	4.069824ms	3.9866ms	8.9979ms
Retrieve from Cache	AcqTokenSilent	100,000	1000	100	8.455385ms	8.011ms	13.2243ms

Future work

We continue to invest in the performance testing. Future work includes increasing the number of metrics (ex. cache size growth) and test cases, running the performance tests on a regular schedule, creating more reports (ex. via App Insights).