Skip to content

Release v0.28.0
Compare
Choose a tag to compare
@lidongze0629 lidongze0629 released this 03 Jul 12:48
· 207 commits to main since this release
v0.28.0
a662886
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.28.0 release. This release contains many important features and improvements to GraphScope FLEX、gsctl command-line utility、Graph Interactive Engine (GIE) and Groot persistent storage.

We highlight the following improvements included in this release:

1. FLEX Interactive

New functionality:

Refactor

  • Clean the data directory if bulk loading fails
  • Adapt to the new schema and data type definition
  • Fix signal handling and compiler launching
  • Upgrade arrow version
  • Refactor and reorganize the stored procedure interface.

2. Enhancements for GIE

Integrated GOpt, a new graph-native query optimizer, into GIE to boost query performance, featuring:

  • Various heuristic optimization rules in Rule-Based Optimization (RBO), such as NotMatchToAntiJoinRule,FilterIntoJoinRule,FilterMatchRule, ExpandGetVFusionRule etc.
  • A new Cost-Based Optimizer (CBO) leveraging high-order statistics for optimal plan selection.
  • Enhanced s-t pathfinding with CBO to enable more efficient bidirectional searches.

More details can be reffered in the paper and doc.

Support some new functionality, including:

  • elementId() to obtain the inner id of graph vertices or edges in Cypher and Gremlin
  • support CASE WHEN in Cypher
  • values() or valueMap() to obtain properties of each element in path in Gremlin

More details can be referred in Cypher Support Doc and Gremlin Support Doc.

3. Enhancements for Groot

  • Support add properties to existed label #3887

Docker Images

# Flex Interactive image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive:0.28.0

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.28.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.28.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.28.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.28.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.28.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.28.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • fix(python): filetype should be in read_options by @siyuan0322 in #3683
  • fix(interactive): Introduce Calcite-Based CBO Optimizer in GIE Compiler by @shirly121 in #3672
  • fix(interactive): Support Intersect with PathExpand in GIE Runtime by @BingqingLyu in #3600
  • feat(learning): Integrate GLTorch with GraphScope in Server-Client Mode with K8S Deployment Supports by @husimplicity in #3624
  • fix(interactive): Support Gremlin Subquery in Calcite-Based IR Layer by @shirly121 in #3636
  • fix(interactive): Support Optional Match in Graph Optimizer by @shirly121 in #3687
  • Pre-release v0.28.0 by @lidongze0629 in #3686
  • fix(interactive): add MovieTest in the new compilation stack by @BingqingLyu in #3690
  • feat(interactive): Support stopping query service by @zhanglei1949 in #3698
  • refactor(interactive): Return the correct http code by @zhanglei1949 in #3700
  • refactor(interactive): Support Arithmetic Operations on Temporal Types by @shirly121 in #3701
  • feat(interactive): support adding a store pod as a backup by @siyuan0322 in #3703
  • fix(interactive): Concurrent compact by @bufapiqi in #3706
  • fix: Add mocking examples for Portal CI test by @lidongze0629 in #3691
  • fix(interactive): Clean data_dir when bulk_loader fails or cancelled by @zhanglei1949 in #3705
  • Update README.md by @wenyuanyu in #3714
  • chore: fix CI problems by @siyuan0322 in #3712
  • refactor(interactive): Introduce MetaDataStore for Interactive and provide file-based implementation by @zhanglei1949 in #3696
  • fix(interactive): Fix Bug of Duplicated Plan Id Generation After Failover by @shirly121 in #3710
  • fix(interactive): Make rust log location conform to LOG_DIR environment variable. by @siyuan0322 in #3716
  • fix(analytical): make clustering support undirected graph by @acezen in #3715
  • fix(learning): Update graphlearn-torch image version by @husimplicity in #3721
  • build(python): adjust the networkx version by @haoxins in #3463
  • fix(interactive): Integrate multi-pod and one-pod charts into one. by @siyuan0322 in #3720
  • feat(python): Unify the graph level load_from and save to API & Bump up vineyard to v0.22.0 by @acezen in #3610
  • fix(interactive): Support Gremlin Match in Calcite-Based IR Layer by @shirly121 in #3694
  • fix(interactive): Identify pk when it is fused with other predicates in IRCore by @BingqingLyu in #3723
  • fix(interactive): add a simpler PhysicalPlanPrinter in Insight Runtime by @BingqingLyu in #3733
  • fix(interactive): Fix Interactive CI test by @zhanglei1949 in #3738
  • refactor(interactive): Add plugin loading test by @zhanglei1949 in #3743
  • refactor(interactive): add plugin into graph meta by @zhanglei1949 in #3744
  • feat(interactive): Adapt to new data type definition by @zhanglei1949 in #3749
  • feat(interactive): Adapt to new unified schema by @zhanglei1949 in #3725
  • fix(interactive): Make sure admin service always return response in application/json by @zhanglei1949 in #3752
  • chore: adapt to unified schema by @siyuan0322 in #3750
  • refactor(interactive): Remove is_builtin in GraphMeta by @zhanglei1949 in #3754
  • fix(interactive): Fix some implementation of MetaService by @zhanglei1949 in #3759
  • refactor(interactive): Support elementId Operator to Get Node Identifier in Cypher by @shirly121 in #3758
  • fix(learning): Automate graphlearn-torch docker-build by @husimplicity in #3753
  • feat(interactive): Introduce Interactive Java/Python SDK by @zhanglei1949 in #3602
  • fix(interactive): add error handling during file operations by @liulx20 in #3763
  • fix(analytical): Fix MessageStrategy customization for java apps by @zhanglei1949 in #3762
  • fix(interactive): Fix Bugs of count() in Gremlin Based on Calcite-IR Layer by @shirly121 in #3751
  • fix(interactive): Fix waiting compiler ready by @zhanglei1949 in #3770
  • fix(interactive): Fix Bugs of Where Subquery in Gremlin by @shirly121 in #3757
  • fix(interactive): put pegasus hosts to configmap.yaml by @siyuan0322 in #3777
  • fix(interactive): Fix interactive doc and image building by @zhanglei1949 in #3767
  • chore: Export realtime write and submitting query metrics in groot by @siyuan0322 in #3776
  • feat(interactive): Implement API get_graph_schema for AdminService by @zhanglei1949 in #3780
  • fix(interactive): Fix signal handling for interactive by @zhanglei1949 in #3764
  • fix(interactive): Support Join of Path Expand in Compiler & Runtime by @shirly121 in #3747
  • refactor(interactive): Add more detail tracer for query handler by @zhanglei1949 in #3782
  • fix(interactive): Do not use otel java agent to reduce export size by @siyuan0322 in #3784
  • fix(interactive): Fix the bug of plugin not being deleted when deleting graph by @zhanglei1949 in #3785
  • fix(interactive): support a general Intersect implementation in GIE Runtime by @BingqingLyu in #3689
  • feat(interactive): Add current running graph meta info in GetServiceStatus API's response. by @zhanglei1949 in #3793
  • fix(interactive): Resolve bulk_loader path discovery issue by @zhanglei1949 in #3795
  • chore: Support connecting to uptrace in kubernetes cluster. by @siyuan0322 in #3796
  • fix(interactive): force flush epoch after reopened rocksdb by @siyuan0322 in #3797
  • fix(interactive): Fix some issues of AdminService by @zhanglei1949 in #3761
  • fix(interactive): refine rust otel export by @siyuan0322 in #3810
  • fix(interactive): Fix bug in ExpandGetVFusionRule, and optimize when GetV has imprecise types by @BingqingLyu in #3804
  • chore: Add estimate memory usage command for gsctl by @siyuan0322 in #3816
  • fix(interactive): Refine generate_sdk.sh for interactive by @zhanglei1949 in #3807
  • fix(interactive): throw unsupported exception for multi-edge patterns in GOpt by @BingqingLyu in #3805
  • fix(interactive): Minor fix for interactive service by @zhanglei1949 in #3827
  • chore: add an example of mpi-operator by @siyuan0322 in #3828
  • fix(interactive): Fix c++ procedure gen and codegen by @zhanglei1949 in #3835
  • feat(interactive): Track the start_time of hqps_service and return in get_service_status by @zhanglei1949 in #3838
  • docs: Update the domain name of GraphScope PlayGround by @lidongze0629 in #3825
  • fix(learning): Adjust setting for ack cluster by @husimplicity in #3842
  • fix(interactive): Remove dummy code by @zhanglei1949 in #3840
  • fix(interactive): Support identical plugin names for distinct plugins across graphs. by @zhanglei1949 in #3846
  • feat(interactive): Refine AppBase and refactor procedure calling, implement call_proc in SDK by @zhanglei1949 in #3815
  • fix(interactive): Fix Bugs of Type Inference in PathExpand by @shirly121 in #3837
  • fix(interactive): Introduce Statistics Interface Abstraction in MetaData by @BingqingLyu in #3843
  • fix(interactive): Fix submitting cpp procedure with raw input format by @zhanglei1949 in #3850
  • fix(interactive): Fix stored_procedure parsing in old-version schema and refactor CI by @zhanglei1949 in #3853
  • fix(interactive): Fix CI and add list_graphs test by @zhanglei1949 in #3858
  • fix(interactive): Fix Bugs of Primary Key Identification in Gremlin Queries by @shirly121 in #3830
  • fix(interactive): Support using string as primary key by @zhanglei1949 in #3857
  • fix(interactive): Fix creation_time error by @zhanglei1949 in #3851
  • fix(interactive): avoid introduce pb to stored procedure by @liulx20 in #3861
  • fix(learning): enable glt in config by @husimplicity in #3852
  • fix(interactive): Fix interactive openapi.yaml by @zhanglei1949 in #3867
  • fix(interactive): Minor fix: service status is stopped after initialized. by @zhanglei1949 in #3871
  • fix(interactive): Install nlohmann/json.hpp as it is needed by procedure compilation by @zhanglei1949 in #3872
  • fix(interactive): Fix Bugs of GOpt Caused By Multiple Filters of Single Vertex by @shirly121 in #3831
  • fix(interactive): add memory level option for interactive server by @liulx20 in #3877
  • fix(interactive): fix the compact thread count by @siyuan0322 in #3882
  • refactor(interactive): Refine Interactive SDK call_proc API design by @zhanglei1949 in #3881
  • Update Latest News on README.md by @yecol in #3885
  • feat(analytical): Support using customized writable as vdata,edata for Giraph apps by @zhanglei1949 in #3873
  • fix(interactive): Introduce API to Get Meta Data From Remote Service by @shirly121 in #3879
  • fix(interactive): persist latest kafka queue offset before stopped by @siyuan0322 in #3800
  • fix(analytical): Fix java sdk support for builtin libs by @zhanglei1949 in #3884
  • feat(interactive): Implement the get_graph_statistics for AdminService by @zhanglei1949 in #3859
  • feat(interactive): Release interactive java/python sdk by @zhanglei1949 in #3886
  • fix(interactive): Support add property with exist label by @bufapiqi in #3887
  • fix(interactive): Fix get_service_status returns "stopped" after service initialized by @zhanglei1949 in #3888
  • fix(interactive): Support Path Expand Until Condition in Gremlin-Calcite by @shirly121 in #3878
  • refactor: reorganize the Coordinator and gsctl to adapt to the unified schema on FLEX Interactive by @lidongze0629 in #3718
  • fix(interactive): support CaseWhen in GIE runtime by @BingqingLyu in #3868
  • fix(interactive): Support Nested Map in Compiler & Runtime to Facilitate Queries like select('a', 'b').by(valueMap()) by @shirly121 in #3839
  • fix(interactive): Fix Type Inference Bugs in Path Expand After Optimization by @shirly121 in #3889
  • fix(interactive): update the integration tests in GOpt-based compilation by @BingqingLyu in #3786
  • refactor(interactive): Upgrade arrow to 15.0.2 by @zhanglei1949 in #3892
  • ci: Auto release gsctl wheel package and Interactive image by @lidongze0629 in #3901
  • fix(python): Decode the k8s node selector and convert it as a dict. by @dashanji in #3477
  • fix(interactive): Fix parsing empty edge_type list when creation graph by @zhanglei1949 in #3902
  • fix(interactive): fix backup conflict checking, refine log recycling by @siyuan0322 in #3909
  • refactor(interactive): Revision interactive from 0.0.3 to 0.3 by @zhanglei1949 in #3905
  • docs(interactive): Add document about column_mapping by @zhanglei1949 in #3912
  • fix(interactive): Check the number of edge properties when creating graph schema by @zhanglei1949 in #3916
  • fix(interactive): Support Statistics in Groot Storage by @BingqingLyu in #3856
  • feat: Start the service when switching graph context on Interactive by @lidongze0629 in #3910
  • fix(interactive): Fix Result Parser Issue of Gremlin select('a', 'b').by(valueMap()) by @shirly121 in #3919
  • fix(interactive): Refine Query Timeout Configuration in GIE Doc by @shirly121 in #3917
  • fix(interactive): Fix Gremlin Alias Issue in GraphBuilder by @shirly121 in #3918
  • fix(interactive): Fix Errors Caused by Conflict Merge With Main by @shirly121 in #3921
  • fix(interactive): consider path direction in path concat by @BingqingLyu in #3920
  • fix(interactive): Fix python sdk packing by @zhanglei1949 in #3928
  • fix(interactive): append query type in graph_db_ic_handler by @liulx20 in #3932
  • fix(interactive): sync_to_file not work when swap called by @liulx20 in #3933
  • fix(interactive): Avoid redundant traversal when building lf_indexer by @liulx20 in #3934
  • fix(interactive): Fixed bug that wasted disk space by @yqylh in #3929
  • fix(interactive): close csr and free memory after dumped by @liulx20 in #3936
  • fix(interactive): Allow duplicate vertices and edge with invalid vertex id in source data by @liulx20 in #3939
  • feat(interactive): supports multithreaded bulk loading by @liulx20 in #3943
  • fix(interactive): Fix a bug in Groot values.yaml by @BingqingLyu in #3955
  • ci: fix some problems in Flex Interactive CI by @lidongze0629 in #3931
  • fix(interactive): Support specifying vertex/edge properties as null by @zhanglei1949 in #3956
  • feat(interactive): add use-mmap-vector and build-csr-in-mem option for bulk loading by @liulx20 in #3957
  • chore: update the invitation link of dingtalk by @siyuan0322 in #3960
  • refactor(interactive): Unify devcontainer.json of graphscope-dev and flex by @zhanglei1949 in #3866
  • fix(interactive): Add data_source in schema_mapping by @zhanglei1949 in #3953
  • Fix typos by @co63oc in #3951
  • fix: support deploy, destroy instance by gsctl by @lidongze0629 in #3963
  • ci: find the gsctl wheel package correctly by @lidongze0629 in #3968
  • fix(interactive): Only allow using of var_char and long_text on edge property for string type by @zhanglei1949 in #3967
  • fix(analytical): Fix ret type of getIntData and getDoubleData for arrow fragment java wrapper by @zhanglei1949 in #3969
  • fix(interactive): Fix ref column creation issues by @zhanglei1949 in #3970
  • fix(interactive): Fix bugs in Gremlin expr Grammar by @shirly121 in #3958
  • fix(interactive): Support NULL Value in expression in GIE Runtime by @BingqingLyu in #3972
  • refactor(interactive): Support timezone info for timestamp type. by @zhanglei1949 in #3975
  • fix(interactive): add tcp write time & fix block mode by @lnfjpt in #3976
  • fix(interactive): Implement HttpIrMetaReader to Get Meta Data From Remote Http Service by @shirly121 in #3908
  • feat(interactive): support multiple properties on edge by @liulx20 in #3962
  • feat(interactive): Support project the properties of elements in Path in GOpt by @BingqingLyu in #3941
  • fix(interactive): Introduce 'mimalloc' as an alternative memory allocator in GIE Backend by @BingqingLyu in #3946
  • feat(interactive): Automatically upload local file when bulk loading by @zhanglei1949 in #3966
  • fix: correct nslookup behavior for Groot helm charts by @lidongze0629 in #3974
  • fix(interactive): Support std::string as member of Any and rename kString to kStringView by @liulx20 in #3979
  • fix: fix VERSION not found in gsctl by @lidongze0629 in #3983
  • fix(interactive): fix some update interface not work by @liulx20 in #3987
  • fix(learning): fix graph-learn CI by @LiSu in #3985
  • fix(CI): differentiate how protobuf is found in mac and linux in graph-learn cmakelist by @LiSu in #3993
  • fix(interactive): handle the case when meta or statistics are not ready in GOpt by @BingqingLyu in #3986
  • refactor(interactive): Add more options to LoadingConfig by @zhanglei1949 in #4000
  • refactor(interactive): Support customizing ports when deploy interactive in local mode and connect via SDK by @zhanglei1949 in #3990
  • fix(interactive): Fix some bug of interactive admin service by @zhanglei1949 in #3938
  • ci: fix analytical engine CI by @lidongze0629 in #3996

New Contributors

Full Changelog: v0.27.0...v0.28.0

Contributors

yecol, LiSu, and 15 other contributors
Assets 2
Loading