Coerce to Csparsematrix for C++ conversion efficiency

[scottgigante]

No description provided.

[scottgigante]

@LuckyMD bumping this

[scottgigante]

Addresses MarioniLab/scran#70

[LuckyMD]

Sorry, haven't gotten around to this. The point about the 2e9 non-zero value limitation... is that sth we need to worry about?

[LuckyMD]

2e9 non-zero values would mean 25k genes and 80k cells max. That doesn't really work for single-cell data, no? If you assum 90% dropout that would be 800k cells. Still not really enough for our 1 million cell case. What do you think about a size limitation for when you cast to dgCmatrix?

[scottgigante]

You're right. Maybe a better solution would be to do this in python.

X = adata.X.T
if scipy.sparse.issparse(X):
    if X.nnz > 2**31-1:
        X = X.tocoo()
    else:
        X = X.tocsc()
ro.global_ev['data_mat'] = X

That way we avoid dgRMatrix which is terrible, and use dgCMatrix when possible.

[LuckyMD]

Could you merge master into this PR? Then I can merge. Looks good now and gives me the same results. We can always troubleshoot if there are storage limitations at some point.

[LuckyMD]

no merge conflicts... so all good.

[scottgigante]

Thanks!

…

On Fri, 25 Sep 2020, 1:00 pm MalteDLuecken, ***@***.***> wrote: no merge conflicts... so all good. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#164 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACA3DX7NTBMLWH3S4J7FNMTSHTEDXANCNFSM4RIJWDYQ> .