first version of tests that reproduce mem leak #2001
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
The tests contain logic that is able to catch large memory leaks by repeating certain operation N times and cleaning mem through garbage collection each time. As the mem leaks is iterrative process the success of approach will be determined by how large the mem leak is. For small mem leaks this process will not be able to capture them unless repeated many many times, which could be very costly in terms of time.
It is advised that when approach is taken and new test is prepared the check procedure parameters to be finetuned for that particular case.. To do so you need to execute the process from command line with "-s" option so that pytest does not hide the output of print statements. See more in the doc of the procedure..
The output that you should expect from precedure should be like:
tart check process for memory leaks
Num iterrations: 30
Maxumum memory growth/lost to the examined process: 4808662528
Maxumum machine memory utilization: 4808662528
Lets pause for 10 secs so GC to kick in
Available memory 19730.07MB/[20688478208]
Process initial RSS 8671.72MB/[9092956160]
Starting watched code ...........
Lets pause for 7 secs so GC to kick in
Iter No[0] : Process did added (or if - number means cleaned) 1014.14MB/[1063407616] . Avail memory: 18793.03MB/[19705921536] Used Mem: 61.0%
Overall stats : Process growth since start 1014.14MB/[1063407616] AVG growth per iter 1014.14MB/[1063407616.0]
Minimum growth so far: 1014.14MB/[1063407616]
Maximum growth so far: 1014.14MB/[1063407616]
Number of times there was 50% drop in memory: 0
Starting watched code ...........
Lets pause for 7 secs so GC to kick in
Iter No[1] : Process did added (or if - number means cleaned) 990.93MB/[1039065088] . Avail memory: 17372.98MB/[18216894464] Used Mem: 63.9%
Overall stats : Process growth since start 2005.07MB/[2102472704] AVG growth per iter 1002.54MB/[1051236352.0]
Minimum growth so far: 1014.14MB/[1063407616]
Maximum growth so far: 2005.07MB/[2102472704]
Number of times there was 50% drop in memory: 0
..........................................................
What does this implement or fix?
Any other comments?
Checklist
Checklist for code changes...