Testing Custos
For the Custos which we built and deployed on our own, we performed Endurance Testing.
Endurance Testing: It is one type of performance testing where the testing is conducted to evaluate the behavior of the system when a significant workload is given continuously
We tested our application with multiple sets of user settings. The application was well until 1000 samples for each request (Ignoring a few minimal failures) so we pushed it harder to 100 users, with 50 loops for 17 services, totaling 85000 samples/requests. The results were not as expected as compared with the failures with 1000 samples one. So we are sharing those results, considering the worst-case scenario.
We tried a setting of 100 users with 50 loops. For user management APIs, the loop around time was 120 seconds, just to test a different setting.
The system was stable at the beginning, but slowly the performance deteriorated.
As we can see above, most of the requests failed for user-management APIs. The update user profile, Get users, reset password, and get user status, were all part of user-management APIs. The same services had a higher failure rate for the already deployed custos.
The response time increased with an increase in requests. Nearly for all services, the response time was higher. The create group service had the highest response time in the end.