Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Usage when calling e.g. btalib.rsi(df) in a loop #11

Open
moritzgun opened this issue Jan 28, 2021 · 5 comments
Open

Memory Usage when calling e.g. btalib.rsi(df) in a loop #11

moritzgun opened this issue Jan 28, 2021 · 5 comments

Comments

@moritzgun
Copy link

I am running a MultiplexSocket on Binance API to get 5 minute klines of around 200 symbols. On every new completed candle I call a "run_technical_analysis" function. Inside that function I calculate indicators for each symbol like this:

period = 14
value = btalib.ema(candlesticks_df.tail(period + 1), period=period) ## Version A
#value = candlesticks_df.Close.tail(14).ewm(span=14).mean() ## Version B

After that I append value.df.tail(1) to a list. On Version A my RAM goes up almost 10 MB every 5 minutes. It seems like the allocated memory for 'value' is not beeing freed. When I use Version B instead there is no such issue.

What can I do to reduce that memory usage?

@reedajohns
Copy link

Same problem on my end!

@drhighliner
Copy link

drhighliner commented Mar 30, 2021

Same problem on my end as well.
Does anybody have a fix for this? Maybe we can create a fork and publish it on pypi ... seems like the developer is not updating this library anymore. :(

@PapoKarlo
Copy link

with btalib.stochastic(df) same problem, memory leak very fast.

@sam1946
Copy link

sam1946 commented Oct 8, 2021

https://stackoverflow.com/questions/13784192/creating-an-empty-pandas-dataframe-then-filling-it

This may solve the issue?
Never append a row to a dataframe

@Davesmacer
Copy link

Every indicator with btalib inside a loop is filling memory, even if we use del to free variables returned. This is a big issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants