Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graphsage data prep for large datasets #12

Open
parhamouni opened this issue Mar 10, 2019 · 1 comment
Open

Graphsage data prep for large datasets #12

parhamouni opened this issue Mar 10, 2019 · 1 comment

Comments

@parhamouni
Copy link

Hello there,
I would like to apply the code for a dataset with a millon nodes, however using the dense matrix would cause a memory error in graphsage data prep. do you have any idea how to fix it?

@matenure
Copy link
Owner

I did not fully understand your question. Not sure whether you want to convert the format of GraphSAGE into the format of FastGCN or inverse.
Anyway, my suggestion is to first check the memory load at each important step and find the bottleneck.
And possible solutions may be:

  1. Change the dense matrix into sparse.
  2. Split your graph into different blocks or batches when doing the data prep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants