Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

N2V in Jupyter Notebook and GPU Memory Issue #99

Open
citypalmtree opened this issue Aug 24, 2020 · 1 comment
Open

N2V in Jupyter Notebook and GPU Memory Issue #99

citypalmtree opened this issue Aug 24, 2020 · 1 comment

Comments

@citypalmtree
Copy link

Hi,

I am currently running N2V in Jupyter Notebook.
In the past, I would run the training module, and restart the kernel to free GPU memory so that I could run the predict module.

Now, I was wondering if I can run both modules simultaneously without having to restart the kernel.
Is there something that terminates TF to conserve GPU memory in the code? something like 'tf.keras.backend.clear_session()'?

I also tried to run the training module in multiprocess, hoping that the session would end and the memory would be freed, but I had to close Jupyter Notebook entirely to free up that GPU in multiprocess.

Any advice would be greatly appreciated!
Thank you.

@tibuch
Copy link
Collaborator

tibuch commented Sep 15, 2020

Hi @citypalmtree,

In my experience running training and predicition sequentially in the same script/notebook works.

The problem of running first the training notebook and then the prediction notebook is, that two kernels are started. But if you have training and prediction running on the same kernel, the GPU memory will be allocated by the same kernel and can be reused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants