Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow bin_leak when transfering some large files #45

Open
ferd opened this issue Oct 20, 2024 · 0 comments
Open

Slow bin_leak when transfering some large files #45

ferd opened this issue Oct 20, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@ferd
Copy link
Owner

ferd commented Oct 20, 2024

Describe the bug
There are slow memory leak due to refc binaries accumulating over large file transfers (large files, with large file counts, in the hundreds to many thousands).

It's not a major thing, but memory usage is easy to spot when running the recon calls for it and looking at the results.

To Reproduce

([email protected])2> recon:bin_leak(5).
[{<0.1278.0>,-950,
  [{current_function,{gen_server,loop,7}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.1302.0>,-867,
  [{current_function,{gen_server,loop,7}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.1288.0>,-647,
  [{current_function,{gen_server,loop,7}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.1280.0>,-497,
  [{current_function,{gen_server,loop,7}},
   {initial_call,{proc_lib,init_p,5}}]},
 {<0.1206.0>,-414,
  [ssl_manager,
   {current_function,{gen_server,loop,7}},
   {initial_call,{proc_lib,init_p,5}}]}]
image

Expected behavior
It would probably be worth it for revault_fsm and its related network transfer workers to run a hibernation from time to time (eg. after a whole sync and after having sync'd file over size N or M files in a row)

Environment:

  • OS:
    • Linux raspberrypi.local 6.6.31+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64 GNU/Linux
    • Darwin airm1.local 24.0.0 Darwin Kernel Version 24.0.0: Tue Sep 24 23:36:26 PDT 2024; root:xnu-11215.1.12~1/RELEASE_ARM64_T8103 arm64
  • Erlang Version: OTP-27.1.1
  • ReVault Release Version: 42b020a

Additional context
This did not cause any actual problem, but I assume over extended time periods it could create system pressure.

Temporary Workaround

  • run the recon gc by hand
  • if it dies, quit and restart, which will resume transfers for all remaining files.
@ferd ferd added triage issue hasn't been assessed yet enhancement New feature or request good first issue Good for newcomers and removed triage issue hasn't been assessed yet labels Oct 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant