Performance Improvements in Home Page Processing

[TeachMeTW]

Enhancing the performance of the dashboard by optimizing UUID processing and refining database queries. These changes aim to reduce load times and improve the overall efficiency of the application.

Key Changes

1. Removed Unnecessary UUID Conversion

   • Bypassed the conversion of UUID strings to UUID objects when not required, streamlining data processing.

2. Optimized MongoDB Aggregation Pipeline

   • Combined multiple $match stages into a single stage to reduce pipeline overhead and accelerate query execution.

3. Enhanced Logging ITemporary)

   • Implemented a log_execution_time decorator to monitor and log the execution time of key functions, aiding in performance tracking and debugging.

Benefits

• Improved Performance: Reduced processing time by eliminating unnecessary operations and optimizing database queries.
• Better Monitoring: Enhanced logging provides clear insights into function execution times, facilitating easier troubleshooting and performance tuning.

Testing

• Functionality: Verified that all dashboard components display correctly and data is accurately processed.
• Performance: Measured execution times before and after optimizations to confirm performance gains.
• Logging: Ensured that execution times are correctly logged for all decorated functions.

[TeachMeTW]

@shankari @JGreenlee I've notified Jack about this already, but right now on prod/staging, the initial overview load times are unbelievably slow, however, on local machine with open access dataset, the load times is only 4 seconds at its worst case. This PR won't really solve anything considering its not reflective of the actual issue. What are your thoughts on next steps?

[shankari]

@TeachMeTW this is why we need to test on staging 😄 We have found earlier that DocumentDB access is ~ 10x slower than mongo on the local laptop, even from an AWS EC2 instance. I am about to merge #124 now, let's see what the logging shows us around optimization before focusing on this change.

[TeachMeTW]

Note the logger is a temporary until emission server func pipeline is completed

[TeachMeTW]

@shankari This has lowered home page loading from 7 sec to 0.5 seconds locally; awaiting staging to see if my fix actually works; the loggers are temporary per stated above. can be removed on a future comit or pr; added to project board as well

[JGreenlee]

It's odd that this would be the bottleneck.
Are the entries of uuid_list already instances of UUID? Or are they strings?

[TeachMeTW]

@JGreenlee uuid_list is already a string; I don't see why we need to convert it to uuids when we are just querying it.

[shankari]

@TeachMeTW do we have the logs deployed to see the "before" performance? If not, I would suggest that we first add logging, so we can get some "before" data, and then deploy the fix for the "after" data

[TeachMeTW]

@shankari see #141 , would you like this implemented on other pages temporarily as well?

[shankari]

I would like to fix e-mission/e-mission-server#986 properly
Then add instrumentation on all pages and access a ~ 5 programs (in addition to staging) to get a representative set of data
Then implement the fixes.

wrt

@shankari @JGreenlee I've notified Jack about this already, but right now on prod/staging, the initial overview load times are unbelievably slow, however, on local machine with open access dataset, the load times is only 4 seconds at its worst case. This PR won't really solve anything considering its not reflective of the actual issue. What are your thoughts on next steps?

We should make it easier to have the local machine be reflective of the real issues so we can focus on the right areas to optimize and not waste our time optimizing areas that are not the real bottlenecks.

To do this, I would suggest:

• look at the logs on stage/prod and the logs on the local machine to see the difference in DB access times
• change the network that we use in the local docker-compose to have higher latency. It is a simulated network anyway, so we can do whatever we want wrt degrading network performance (e.g. https://medium.com/@kazushi/simulate-high-latency-network-using-docker-containerand-tc-commands-a3e503ea4307)

Please get the instrumentation done first, so that we can collect the before data and then use that to motivate these additional fixes.