Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache Expiry and Persistence for "load_image_async" Function #36

Open
munishkhatri720 opened this issue Jun 24, 2024 · 5 comments
Open

Comments

@munishkhatri720
Copy link

Does this will not increase memory usage of the application in which easy_pil is used ?
image

I have an api which builds musiccard every second and I noticed my vps memory usage touched 1gb in just few minutes
image

@munishkhatri720 munishkhatri720 changed the title Issue related to load_image_async function Cache Expiry and Persistence for load_image_async Function Jun 24, 2024
@munishkhatri720 munishkhatri720 changed the title Cache Expiry and Persistence for load_image_async Function Cache Expiry and Persistence for "load_image_async" Function Jun 24, 2024
@shahriyardx
Copy link
Owner

Are you closing the editor after generating images ?

@munishkhatri720
Copy link
Author

munishkhatri720 commented Sep 2, 2024

Yes, I am closing the editor after modifying the image. But my question is, why is the cache TTL set to 12 hours? I used easy_pil with FastAPI, which handles 22,000 images every day, but the API's RAM usage reached 7 GB in a single day. I think the load_image_async function is caching images for too long, which is causing the high RAM usage.

@shahriyardx
Copy link
Owner

shahriyardx commented Sep 2, 2024

So, do you think removing cache is better so you can implement your own caching strategy?
Also load_image loads the image from a link then opens it using Image.open() you need to close this too after the usage of the image is done

editor = Editor()

img = await load_image_async(url)
editor.paste(img, ...)
img.close()

bytes = editor.image_bytes
editor.close()

Let me know if this strategy helps reducing ram usage, ultimately all the images that you open must be closed to improve performance

@munishkhatri720
Copy link
Author

So, do you think removing cache is better so you can implement your own caching strategy?
Also load_image loads the image from a link then opens it using Image.open() you need to close this too after the usage of the image is done

editor = Editor()

img = await load_image_async(url)
editor.paste(img, ...)
img.close()

bytes = editor.image_bytes
editor.close()

Let me know if this strategy helps reducing ram usage, ultimately all the images that you open must be closed to improve performance

Yes, remove the cache strategy and allow people to implement their own. Otherwise, many users won't see the source code and will start using it in production. Later, they may complain about memory leaks. Alternatively, you could implement an LFU-based caching system that caches up to a set limit.

@shahriyardx
Copy link
Owner

ok I have removed the caching in version 0.4.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants