diff --git a/docs/interactive_engine/benchmark_tool.md b/docs/interactive_engine/benchmark_tool.md index c077f9491896..c8daf1f6f311 100644 --- a/docs/interactive_engine/benchmark_tool.md +++ b/docs/interactive_engine/benchmark_tool.md @@ -25,7 +25,7 @@ The program uses a round-robin strategy to iterate all the **enabled** queries w - bin - bench.sh // script for running benchmark for queries - collect.sh // script for collecting benchmark results -- config +- config - interactive-benchmark.properties // configurations for running benchmark - data - substitution_parameters // query parameter files using to fill the query templates @@ -43,7 +43,7 @@ and the queries with the prefix _job_ are the implementation of JOB Benchmark. The gremlin queries should be with suffix _.gremlin_, and cypher queries should be with suffix _.cypher_. The corresponding parameters (factor 1) for LDBC queries are generated by [LDBC official tools](http://github.com/ldbc/ldbc_snb_datagen). -### Building +### Building the benchmark Build benchmark program using Maven: @@ -56,8 +56,13 @@ and you can use deploy the package to anywhere could connect to the gremlin endp ### Running the benchmark +You can unzip builded _target/benchmark-0.0.1-SNAPSHOT-dist.tar.gz_, and run the benchmark. + ```bash -./bin/bench.sh # run the benchmark program with the provided properties +cd target +tar -xvf gaia-benchmark-0.0.1-SNAPSHOT-dist.tar.gz +cd gaia-benchmark-0.0.1-SNAPSHOT +./bin/bench.sh # run the benchmark program. You can also modify running configurations in config/interactive-benchmark.properties ``` With the example configuration file ``example/job_benchmark.properties``, which compares GraphScope-GIE and KuzuDB while executing the JOB Benchmark, the example of results are as follows: @@ -88,7 +93,7 @@ System: KuzuDB; query count: 35; execute time(ms): xxx qps: xxx ./bin/collect.sh # run the result collection program to collect the results and generate a performance comparison table ``` -Based on the benchmark results, the collected data and the final performance comparison table are as follows: +Furthermore, based on the benchmark results, the collected data and the final performance comparison table are as follows: | QueryName | GIE Avg | GIE P50 | GIE P90 | GIE P95 | GIE P99 | GIE Count | KuzuDb Avg | KuzuDb P50 | KuzuDb P90 | KuzuDb P95 | KuzuDb P99 | KuzuDb Count | diff --git a/interactive_engine/benchmark/README.md b/interactive_engine/benchmark/README.md index 853c6c26dec7..4df817b5d0f3 100644 --- a/interactive_engine/benchmark/README.md +++ b/interactive_engine/benchmark/README.md @@ -38,7 +38,10 @@ and you can use deploy the package to anywhere could connect to the endpoint (wh ### Running the benchmark ```bash -./bin/bench.sh # run the benchmark program with the provided properties +cd target +tar -xvf gaia-benchmark-0.0.1-SNAPSHOT-dist.tar.gz +cd gaia-benchmark-0.0.1-SNAPSHOT +./bin/bench.sh # run the benchmark program. You can also modify running configurations in config/interactive-benchmark.properties ``` With the example configuration file ``example/job_benchmark.properties``, which compares GraphScope-GIE and KuzuDB while executing the JOB Benchmark, the results are as follows: ``` @@ -66,7 +69,7 @@ System: KuzuDB; query count: 35; execute time(ms): xxx qps: xxx ```bash ./bin/collect.sh # run the result collection program to collect the results and generate a performance comparison table ``` -Based on the benchmark results, the collected data and the final performance comparison table are as follows: +Furthermore, based on the benchmark results, the collected data and the final performance comparison table are as follows: And the comparison result after collection is as follows: | QueryName | GIE Avg | GIE P50 | GIE P90 | GIE P95 | GIE P99 | GIE Count | KuzuDb Avg | KuzuDb P50 | KuzuDb P90 | KuzuDb P95 | KuzuDb P99 | KuzuDb Count | diff --git a/interactive_engine/benchmark/queries/cypher_queries/ic/gie/ldbc_query_11.cypher b/interactive_engine/benchmark/queries/cypher_queries/ic/gie/ldbc_query_11.cypher index 30f176e6959e..0bad88558760 100644 --- a/interactive_engine/benchmark/queries/cypher_queries/ic/gie/ldbc_query_11.cypher +++ b/interactive_engine/benchmark/queries/cypher_queries/ic/gie/ldbc_query_11.cypher @@ -1,6 +1,6 @@ -MATCH (person:PERSON {id: $personId})-[:KNOWS*1..3]-(friend:PERSON)-[workAt:WORKAT]->(company:COMPANY)-[:ISLOCATEDIN]->(:COUNTRY {name: '$countryName'}) +MATCH (person:PERSON {id: $personId})-[:KNOWS*1..3]-(friend:PERSON)-[workAt:WORKAT]->(company:ORGANISATION)-[:ISLOCATEDIN]->(:PLACE {name: '$countryName'}) WHERE person <> friend - and workAt.workFrom < $workFromYear + and workAt.workFrom < $workFromYear RETURN DISTINCT friend.id AS personId, friend.firstName AS personFirstName,