Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Providing a dump version of Hetionet #43

Open
schatzkara opened this issue Sep 19, 2021 · 11 comments
Open

Providing a dump version of Hetionet #43

schatzkara opened this issue Sep 19, 2021 · 11 comments

Comments

@schatzkara
Copy link

Hi! Would it be possible to get a .dump version of Hetionet? I use multiple databases in neo4j and have found that this is easiest for loading new databases. By dump version, I mean one created via the neo4j-admin dump command which has file extension .dump (I can give more details if necessary)

Thanks!

@dhimmel
Copy link
Member

dhimmel commented Sep 20, 2021

Ah the neo4j-admin dump command does look ideal. This command wasn't around I don't think when we initially started providing the neo4j exports.

Note that the production database is still using Neo4j 3.5 (#32), so that might also create issues. Will need to check whether neo4j-admin dump exists for 3.5 or look into upgrading again.

I won't have time immediately to look into this, so let us know if you're able to create the dump which we could point other users towards.

@schatzkara
Copy link
Author

Thanks for the quick response!

I currently use neo4j 4.2 and have been unable to run Hetionet without the dump file. I am trying to avoid having two versions of neo4j running at once, but could certainly try that.

I am going to keep trying to find work arounds and will certainly let you know if anything works!

@dhimmel
Copy link
Member

dhimmel commented Sep 21, 2021

I am trying to avoid having two versions of neo4j running at once, but could certainly try that.

Ideally we could run the export in the Docker container, so its reproducible and doesn't install an extra version of neo4j on the host system.

@yingding
Copy link

I would be interested in a dump version too. I am currently using neo4j 4.1.9 also.

@dhimmel
Copy link
Member

dhimmel commented Jul 10, 2022

I'm able to create the dump file with the following:

docker run \
  --name=hetionet-container \
  --rm \
  --publish=7474:7474 \
  --publish=7687:7687 \
  --volume=$HOME/neo4j/hetionet-data:/data \
  --volume=$HOME/neo4j/hetionet-logs:/logs \
  dhimmel/hetionet:hetionet-v1.0_neo4j-3.5.12 \
  neo4j-admin dump --to=/data/hetionet-v1.0-neo4j.dump

Will look into whether this dump from neo4j 3.5.12 can be loaded into neo4j 4+.

dhimmel added a commit that referenced this issue Jul 10, 2022
@dhimmel
Copy link
Member

dhimmel commented Jul 10, 2022

Okay dump file located at hetionet-v1.0-neo4j.dump. Download link at:

https://github.com/hetio/hetionet/raw/030e44430f168d934c44860ad5a9512deb7c7175/hetnet/neo4j/hetionet-v1.0-neo4j.dump

I haven't tested neo4j-admin load and whether neo4j 4 can load this file. But hopeful it works.

CC @ldnam. Perhaps you can try importing this file via neo4j desktop and see if it works!

@DimitrisAlivas
Copy link

Hi, first of all thank you for your work on HetioNet.
I am trying to use the dump you shared, load it in my local Neo4j Desktop version in a new DBMS.

However, I am stuck. It throws some error (a pretty big log file).
Is there any update on this?

@dhimmel
Copy link
Member

dhimmel commented Jan 25, 2023

However, I am stuck. It throws some error (a pretty big log file)

Can you provide the error message and and note what version of Neo4j Desktop you're using? Put the log in a code block and if it is too large I'll edit your comment to use <details> so it can be minimized.

@DimitrisAlivas
Copy link

DimitrisAlivas commented Jan 25, 2023

Thanks for the blazing fast response.

Neo4j Desktop: Version 1.5.7 (1.5.7.102)

Neo4j Community version: 4.4.16

Expand for Neo4j Desktop Log
[2023-01-25 15:47:29.188] [info]  Neo4j Download check
[2023-01-25 15:47:36.983] [info]  Using Java version 11 for Neo4j 4.0.0
[2023-01-25 15:47:40.466] [error] Selecting JVM - Version:11.0.17+8-LTS, Name:OpenJDK 64-Bit Server VM, Vendor:Azul Systems, Inc.
Not a valid Neo4j archive: /private/var/folders/xt/z_hpsw257x1b02ydd1wl1zhh0000gn/T/5081ba97f0fdb2535e6b7a29cd8ee30c/dumps/neo4j.dump

[2023-01-25 15:47:40.586] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-dc15d3f1-886b-4aff-8fbb-b89b19a7a9cb not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async neo4jConfFolderPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9497:69)
    at async neo4jConfPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9507:63)
    at async Object.getConfig (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:10290:22)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:197:17
    at async Promise.all (index 0)
[2023-01-25 15:47:40.586] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-2355a562-9f98-47ba-864d-b5060b0c9c38 not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async neo4jConfFolderPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9497:69)
    at async neo4jConfPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9507:63)
    at async Object.getConfig (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:10290:22)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:197:17
    at async Promise.all (index 1)
[2023-01-25 15:47:40.586] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-dc15d3f1-886b-4aff-8fbb-b89b19a7a9cb not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async neo4jConfFolderPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9497:69)
    at async neo4jConfPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9507:63)
    at async Object.getConfig (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:10290:22)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:197:17
    at async Promise.all (index 0)
[2023-01-25 15:47:40.587] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-2355a562-9f98-47ba-864d-b5060b0c9c38 not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async neo4jConfFolderPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9497:69)
    at async neo4jConfPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9507:63)
    at async Object.getConfig (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:10290:22)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:197:17
    at async Promise.all (index 1)
[2023-01-25 15:47:40.587] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-dc15d3f1-886b-4aff-8fbb-b89b19a7a9cb not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async neo4jConfFolderPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9497:69)
    at async neo4jConfPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9507:63)
    at async Object.getConfig (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:10290:22)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:197:17
    at async Promise.all (index 0)
[2023-01-25 15:47:40.587] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-2355a562-9f98-47ba-864d-b5060b0c9c38 not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async neo4jConfFolderPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9497:69)
    at async neo4jConfPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9507:63)
    at async Object.getConfig (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:10290:22)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:197:17
    at async Promise.all (index 1)
[2023-01-25 15:47:40.595] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-dc15d3f1-886b-4aff-8fbb-b89b19a7a9cb not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async transformLocalDbmsConfiguration (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:54:15)
    at async transformDbmsToGraph (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:147:31)
    at async Promise.all (index 0)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:226:21
    at async Promise.all (index 0)
[2023-01-25 15:47:40.596] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-2355a562-9f98-47ba-864d-b5060b0c9c38 not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async transformLocalDbmsConfiguration (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:54:15)
    at async transformDbmsToGraph (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:147:31)
    at async Promise.all (index 0)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:226:21
    at async Promise.all (index 1)
[2023-01-25 15:47:40.598] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-dc15d3f1-886b-4aff-8fbb-b89b19a7a9cb not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async transformLocalDbmsConfiguration (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:54:15)
    at async transformDbmsToGraph (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:147:31)
    at async Promise.all (index 0)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:226:21
    at async Promise.all (index 0)
[2023-01-25 15:47:40.599] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-2355a562-9f98-47ba-864d-b5060b0c9c38 not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async transformLocalDbmsConfiguration (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:54:15)
    at async transformDbmsToGraph (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:147:31)
    at async Promise.all (index 0)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:226:21
    at async Promise.all (index 1)
[2023-01-25 15:47:40.601] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-dc15d3f1-886b-4aff-8fbb-b89b19a7a9cb not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async transformLocalDbmsConfiguration (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:54:15)
    at async transformDbmsToGraph (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:147:31)
    at async Promise.all (index 0)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:226:21
    at async Promise.all (index 0)
[2023-01-25 15:47:40.602] [warn]  Failed to get DBMS path through Relate Error: Relate DBMS with tag neo4j-desktop-dbms-database-2355a562-9f98-47ba-864d-b5060b0c9c38 not found
    at /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13613:15
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/monad.js:102:46)
    at Maybe.getOrElse (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/node_modules/@relate/types/dist/monads/primitive/maybe.monad.js:92:22)
    at getRelateDbms (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:13612:24)
    at async neo4jPath (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:9485:22)
    at async transformLocalDbmsConfiguration (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:54:15)
    at async transformDbmsToGraph (/Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:147:31)
    at async Promise.all (index 0)
    at async /Applications/Neo4j Desktop.app/Contents/Resources/app.asar/dist/main.prod.js:226:21
    at async Promise.all (index 1)
[2023-01-25 15:48:04.087] [info]  Online check request: https://dist.neo4j.org/neo4j-desktop/win/latest.yml
[2023-01-25 15:48:04.219] [info]  Online check response: 200 version: 1.5.7
files
[2023-01-25 15:48:44.090] [info]  Online check request: https://dist.neo4j.org/neo4j-desktop/win/latest.yml
[2023-01-25 15:48:44.262] [info]  Online check response: 200 version: 1.5.7
files

This is the main.log of Neo4J Desktop when I try to "Create new DBMS from dump"

Screenshot 2023-01-25 at 15 50 49

I also tried creating a fresh DBMS and then importing the dump without success.
I also tried with neo4j-admin load without much success.

I'm trying to share the logs specific to this instance but they appear to be empty due to some permission issue..
Trying to resolve this issue and get back to you.

Is there any well-known issue I'm unaware of? Is it the version of Neo4j ?

Any help would be greatly appreciated.

@dhimmel
Copy link
Member

dhimmel commented Jan 25, 2023

I think the "Neo4j DBMS logs" is where the actual error message for loading the dump file is. Is this the file you're referring to as unavailable due to:

the logs specific to this instance ... appear to be empty due to some permission issue.

I'm pretty sure the issue is that the dump was created in neo4j 3.5.12 and the dump files are not compatible across major releases. Ideally, we can create a dump for Neo4j 4, but no timeframe on that.

@DimitrisAlivas
Copy link

Yes indeed, that's the file I haven't been able to check yet.

I'm pretty sure the issue is that the dump was created in neo4j 3.5.12 and the dump files are not compatible across major releases.

Yeah, I had similar thoughts/worries. Perhaps, one way out of converting hetionet once again for later versions of Neo4j would be to use the docker image (?)

If I end up doing the conversion for later versions like 4.*.*, I'll open a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants