Memory consumption for DirCacheCheckout

[arysin]

Conversation moved from https://www.eclipse.org/forums/index.php/t/1116555/

I have a Git repo that's about 25M in size:

$ du -sb prj1
26878375        prj1

It has about 6700 files in about 1500 folders.

git clone uses about 17M of RAM max:

$ /usr/bin/time -v git clone mygitserver.com/prj1
...
        Maximum resident set size (kbytes): 16532

But jgit (7.0.0) runs out of memory on my microservice with -Xmx256m (it does so even if I do -Xmx350m)

It seems that DirCacheCheckout holds 57M in updated field, and 63M in builder.

Baseline for the microservice is about 85M, so it looks that to checkout this repo jgit needs more than ~160M.

Is there a way to improve memory usage or tell jgit a cache limit??

[arysin]

From @msohn :

How can the updated hashmap end up with 271k elements when checking out 6.7k files (folders aren't tracked in the git index) ?
Can you retry and create a heap dump when the OOM strikes and analyze that using memory analyzer ?

[arysin]

I've analyzed this in MAT and it looks like ~270k elements in "updated" and ~340k elements in "builder" makes sense, as that includes both alive and deleted files across all branches (also dead and alive). So from the current point in time for the end user, they see only 6.7k files but Git repo contains much bigger history.

On the implementation side I have this:

On original checkout (when local repo does not exist) I do

cloneCommand.
                .setCloneAllBranches(false)
                .setDepth(1);

Not sure if those arguments help this case much. But once the folder exists, I just do
fetchCommand.call()
and then if branch is the same I do ResetCommand, but if it's not then I do
git.checkout() .setForceRefUpdate(true)
From what I can tell the out of memory happens for this last use case.

If there's better approach to do this on my side I'd appreciate any help.