Skip to content

Release v0.26.0
Compare
Choose a tag to compare
@lidongze0629 lidongze0629 released this 25 Jan 03:10
· 478 commits to main since this release
v0.26.0
6db110e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.26.0 release. This release contains many important features and improvements to GraphScope FLEX and Graph Interactive Engine (GIE), including secondary instance and type inference in GIE, loading from distributed data source and Macs deployment in FLEX Interactive.

We highlight the following improvements included in this release:

1. FLEX Interactive

  • Support use GraphScope Interactive on macos. Download the latest release to give it a shot interactive-latest
  • Support bulk loading with odps table as data source. Here is an example
graph: dd_graph
loading_config:
  data_source:
    scheme: odps  # file, odps
  import_option: init # init, overwrite
  format:
    type: arrow
vertex_mappings:
  - type_name: Person  # must align with the schema
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
  column_mappings:
      - column:
          index: 0  # can be omitted if the index is the same as the property index
          name: user_id  # can be omitted if the name is not known
        property: id
edge_mappings:
  - type_triplet:
      edge: Knows
      source_vertex: Person
      destination_vertex: Person
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
    source_vertex_mappings:
      - column:
          index: 0
          name: src_user_id
        property: id
    destination_vertex_mappings:
      - column:
          index: 1
          name: dst_user_id
        property: id

2. Enhancements for GIE

  • Support secondary instance in Groot. It enables user open another read-only instance along with the original instance, providing additional read qps to users.

    • How to use: Set secondary.enabled=true in helm charts to enable secondary mode. Other options could remain unchanges with the primary instance.
    • Note in secondary instance it could only be queried upon, the modification of schema and data should be routed to primary instance.
    • See more details in the doc

  • Support automatic type inference in graph queries, which can affect query execution in two ways:

    • Checking if vertex and edge types in the graph conform to user-defined schemas 
      # Now compiler can infer that 'kno' is a nonexistent edge type and returns error
Match (a:person)-[b:kno]->(c) Return a, b, c;
    • Inferring vertex and edge types in the graph. 
      # Now compiler can infer a and b must both be person types, and execution can be more efficient
Match (a)-[:knows]->(b) Return labels(a), labels(b)
=> Match (a:person)-[:knows]->(b:person) Return labels(a), labels(b)

See more details in the doc

3. Other enhancements and bug fixes

  • Introduce graph_algo as another example graph. https://github.com/alibaba/GraphScope/tree/main/flex/interactive/examples/graph_algo.
  • Support String DT_STRING as edge property type.
  • Refactor Flex/CMakeLists.txt to support use installed flex via find_package(flex).
  • Fix decode error in Groot
  • Fix memory leak in Pegasus and data race in Groot
  • Fix GIE peers_contains bugs in count operator in subtasks
  • Fix Pegasus unexpect result of operator aggregate() + iterate()
  • Refine Log Printing Messages in Compiler
  • Fix bugs of Groot bulk loading when it retry to download source files
  • Fix bugs in GIE for edge filtering

Docker Images

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.26.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.26.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.26.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.26.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.26.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.26.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • ci: fixes failure during pushing image and pre-release v0.26.0 by @lidongze0629 in #3339
  • fix(interactive): Fix decode error in groot by @siyuan0322 in #3340
  • fix(interactive): Check vertex existence whenever queried by @siyuan0322 in #3343
  • feat(interactive): Support more data types for vertex/edge property by @zhanglei1949 in #3346
  • feat(interactive): Support fold_partiton_by_key by @lnfjpt in #3324
  • fix(interactive): fix a memory leak bug in IR Core by @BingqingLyu in #3349
  • fix(interactive): replace ok_or() by ok_or_else() for lazy evaluation by @BingqingLyu in #3350
  • feat(flex): support LFIndexer resize by @liulx20 in #3351
  • refactor(interactive): Refactor CSVFragmentLoader to remove redundant code by @zhanglei1949 in #3352
  • fix(interactive): Fix support for PathExpand for multiple triplets. by @zhanglei1949 in #3335
  • fix(interactive): Remove Unnecessary JNA Structure Creation to Save Memory Consumption by @shirly121 in #3354
  • fix(interactive): extension use hash(srcId, dstId, edgeLabelId, edgePks) instead of increase eid by @bufapiqi in #3357
  • fix(interactive): Comparison between std::shared_ptr<arrow::DataType> should use Equals() by @zhanglei1949 in #3361
  • chore: fixes the missing tzdata issue in graphscope ubuntu-22.04 based images by @sighingnow in #3363
  • chore: make sure tzdata can be installed by @sighingnow in #3365
  • fix(interactive): fix memory leak in pegasus by @lnfjpt in #3367
  • fix(interactive): add some optimizer microbenchmark queries by @BingqingLyu in #3369
  • fix(interactive): Fix memory leak in pegasus and data race in groot by @siyuan0322 in #3371
  • chore: Add writer interface for groot by @siyuan0322 in #3373
  • fix(interactive): Add a new release-with-debug-info profile by @siyuan0322 in #3372
  • fix(interactive): fix the bug in filtering after intersection in GIE Runtime in distributed computation by @BingqingLyu in #3359
  • feat(interactive): Add Rate Limiter for Control Flow by @shirly121 in #3368
  • refactor(interactive): Improve the code of checking if a given label sets contain all labels in the schema by @longbinlai in #3374
  • feat(interactive): Implement admin http service for hqps engine by @zhanglei1949 in #3322
  • fix(interactive): Support Start Tag for Graph Operators in Compiler by @shirly121 in #3376
  • fix(interactive): Fix documentation for interactive. by @zhanglei1949 in #3384
  • docs: Fix GIE Benchmark Readme bug by @JackyYangPassion in #3389
  • feat(interactive): Enable replay WAL records from offset of timestamp in groot by @siyuan0322 in #3385
  • refactor(interactive): flex storage with mmap array by @liulx20 in #3356
  • fix(interactive): event message error by @lnfjpt in #3380
  • fix(interactive): fix peers_contains bugs in count operator in subtasks by @lnfjpt in #3390
  • fix(interactive): fix unexpect result of operator aggregate() + iterate() by @lnfjpt in #3391
  • fix(interactive): Fix Start Time Metric in Compiler Log by @shirly121 in #3393
  • fix(interactive): Refine Log Printing Messages in Compiler by @shirly121 in #3381
  • feat(interactive): Introduce explicit barrier for actors when switching query service to a different graph by @zhanglei1949 in #3395
  • feat(interactive): Add support for loading graph with odps table as data source by @zhanglei1949 in #3305
  • refactor(interactive): Support Logical Operator within in Index Predicate by @shirly121 in #3398
  • feat(interactive): support string as edge property type by @liulx20 in #3403
  • fix(interactive): Pass acid by @liulx20 in #3415
  • fix(python): fixes the result validation for edge sets by @sighingnow in #3421
  • fix(coordinator): Fix coordinator monitor bug by @JackyYangPassion in #3420
  • chore: bump vineyard version to 0.18.2 by @siyuan0322 in #3392
  • fix(python): fixes the error handling of starting analytical engine process by @sighingnow in #3422
  • ci: skip some GIE test cases due to the upgrade of vineyard by @sighingnow in #3424
  • fix(interactive): Remove gcc option -mno-avx512f by @zhanglei1949 in #3429
  • feat(interactive): Import VARCHAR to GraphScope interactive by @zhanglei1949 in #3404
  • refactor(interactive): Remove dummy config incoming/outgoing_edge_strategy by @zhanglei1949 in #3434
  • fix(interactive): Fix bugs of Groot Bulk Load When Retrying to Download Source Files by @shirly121 in #3438
  • feat(analytical): Support extend label data for graph in eager mode. by @SighingSnow in #3288
  • ci: bump the required glt version to fixes MacOS CI failure by @sighingnow in #3440
  • feat(interactive): Add helm chart for Interactive by @zhanglei1949 in #3387
  • fix(interactive): fix isnull in expression evaluation by @BingqingLyu in #3423
  • ci: fixes the destination dir of ossutil64 download for helm repo index by @sighingnow in #3445
  • chore: probing analytical engine in coordinator shouldn't be warnings by @sighingnow in #3446
  • feat(interactive): Introduce graph_algo as another example graph by @zhanglei1949 in #3344
  • feat(interactive): Pegasus return status codes of rpc by @lnfjpt in #3427
  • fix(interactive): Support using interactive on macos by @zhanglei1949 in #3416
  • feat(interactive): Implemented memory-only mode for storage of Interactive. by @luoxiaojian in #3444
  • fix(interactive): Fix bugs When Counting on Multiple Columns by @shirly121 in #3442
  • fix(interactive): Convert Label Condition to Type Constraint by @shirly121 in #3441
  • fix(interactive): Fix the Invalid cross-device link error occurring with copy_file_range in the Docker environment. by @liulx20 in #3454
  • fix(interactive): Fix bug in ODPSFragmentLoader by @zhanglei1949 in #3460
  • fix(interactive): fix dead lock caused by write lock by @lnfjpt in #3459
  • fix(interactive): fix test error of groot and unsafe static reference by @longbinlai in #3469
  • refactor(interactive): revert encoding of groot by @longbinlai in #3471
  • feat(interactive): Introduce Graph Type Inference in Compiler by @shirly121 in #3375
  • refactor(flex): Refactor CMakeLists.txt to support be imported by find_package by @luoxiaojian in #3478
  • feat(interactive): Support secondary instance of groot by @siyuan0322 in #3479
  • fix(analytical): fix the louvain algorithm that the result is not consistent with #183 by @acezen in #3472
  • feat(flex): Implemented compaction. by @luoxiaojian in #3482
  • feat(flex): Implemented basic support for hugepages. by @luoxiaojian in #3489
  • fix(analytical): Fix vineyard image is not set correctly by @siyuan0322 in #3491
  • fix(analytical): Fix the rapidjson find_package by @acezen in #3492
  • fix(python): prevent the GAE been blocked if GIE failed to generate subgraphs during subgraph queries by @sighingnow in #3499
  • fix(core): enable date and time types support in vineyard-graph (and GIE) by @sighingnow in #3498
  • Try fixes macos nightly CI by @sighingnow in #3502
  • fix(interactive): fix a bug in runtime for edge filtering by @BingqingLyu in #3494
  • fix(interactive): Remove assert & unwrap used in pegasus by @lnfjpt in #3488
  • ci: Disable to publish images and packages on the aarch64 platform by @lidongze0629 in #3503

New Contributors

Full Changelog: v0.25.0...v0.26.0

Contributors

longbinlai, sighingnow, and 12 other contributors
Assets 4
Loading