-
Notifications
You must be signed in to change notification settings - Fork 4
Optimization Opportunities
Jerry Morrison edited this page Mar 13, 2019
·
5 revisions
Potential opportunities to make the software run significantly faster and/or in less memory:
- [Fixed in newer releases of Fireworks?] Fireworks commands like
lpad get_fws
can take 10-20 seconds (!), apparently loading and serializing the tasks (classes or instances?) in order to Register your Firetask, and at least one of those tasks is making it recurse over much of the codebase.
It might save a bunch of time to register the firetasks another way: "modify the USER_PACKAGES variable of the FW config to include the package for where to find the Firetask..." - Jenkins builds can run out of memory.
- Running all the analysis scripts can run out of memory. Are some of them leaking large objects?
The Python garbage collector frees objects in reference cycles, but it won't free objects that have__del__
methods because it can't determine a safe order to destroy them.
Here's a way to detect them:
gc.set_debug(gc.DEBUG_UNCOLLECTABLE | gc.DEBUG_INSTANCES | gc.DEBUG_OBJECTS)
... # run an analysis plot
gc.collect()
# gc.garbage is a list of objects the collector found to be unreachable but
# couldn't free since they have __del__ methods.
print "***DEBUG: gc.garbage=", gc.garbage
gc.set_debug(0)
- polymerize.py has more optimization opportunities.
- mc_complexation.pyx might benefit from using memory views onto NumPy arrays instead of the older Cython access.