feat: GraphQL API

[cvauclair]

Add a GraphQL API on top of Neo4j

Motivation

Cypher queries will get more complicated as new features (e.g.: versioning, etc.) are added. Adding a query layer on top of the database will make this more manageable.

Proposal

Create a new GraphQL API service as part of the indexer stack that will serve as the main interface to query data from the KG.

Below the proposed simple generic schema for the GraphQL API

# Represents arbitrary JSON data
scalar Attributes;

type Entity {
  id: String
  types: Vec<String>
  # other system attributes
  ...
  # Entity attributes
  attributes: Attributes
  relations: [Relation!]! 
}

type Relation {
  id: String
  type: String
  # other system attributes
  ...
  # Entity attributes
  attributes: Attributes
  from: Entity
  to: Entity
}

type Query {
  node(id: String): Node

  relation(id: String): Relation
}

Example Query

query {
  entity(
    id: "98wgvodwzidmVA4ryVzGX6",
  ) {
    id
    types
    attributes
    relations {
      type
      attributes
      to {
        id
        types 
        attributes
      }
    }
  }
}

Note: This is roughly equivalent to the following cypher query

MATCH (n {id: "98wgvodwzidmVA4ryVzGX6"}) -[r]-> (m) 
RETURN n, r, m

Here is the response:

{
  "data": {
    "entity": {
      "id": "98wgvodwzidmVA4ryVzGX6",
      "types": [
        "9u4zseS3EDXG9ZvwR9RmqU"
      ],
      "attributes": {
        "GG8Z4cSkjv8CywbkLqVU5M": "Person Posts Page Template"
      },
      "relations": [
        {
          "type": "DD9FKRZ3XezaKEGUszMB3r",
          "attributes": {
            "gEfvT3cW16tyPmFEGA9bp": "a0"
          },
          "to": {
            "id": "E3jboNrTeuopjKgJ45ykBd",
            "types": [],
            "attributes": {
              "GG8Z4cSkjv8CywbkLqVU5M": "Posts page"
            }
          }
        },
        {
          "type": "XbVubxtJCexLsmEhTUKPG",
          "attributes": {
            "gEfvT3cW16tyPmFEGA9bp": "a0"
          },
          "to": {
            "id": "68AAMfbZkZGP1Qb1FxoZ8A",
            "types": [
              "PnQsGwnnztrLNRCm9mcKKY"
            ],
            "attributes": {
              "GG8Z4cSkjv8CywbkLqVU5M": "Posts"
            }
          }
        }
      ]
    }
  }
}

[yanivtal]

Nice this looks great! Only 3 points:

• I think we should use Entity instead of Node for consistency with GRC-20
• Relations can also have Relations as well as Attributes
• Should we add the implicit properties to the schema for both Entity and Relation? In addition to Type(s) which you have, that would be: Name, Description, Cover

Otherwise looks great!

[cvauclair]

I think we should use Entity instead of Node for consistency with GRC-20

Will change!

Relations can also have Relations as well as Attributes

This is a bit tricky since we "rolled up" relations as edges in Neo4j. This would mean going back to representing a relation as a node in Neo4j with edges FROM_ENTITY and TO_ENTITY. It's totally doable though and I guess since users won't be interfacing with Neo4j directly then we don't care so much about "rolled up" relations anymore

Should we add the implicit properties to the schema for both Entity and Relation? In addition to Type(s) which you have, that would be: Name, Description, Cover

Since those properties are not enforced by the indexer, users can define entities without setting those properties. If we make them explicit in the schema, the question arises as to how we'd handle cases when they are missing?

[baiirun]

Since those properties are not enforced by the indexer, users can define entities without setting those properties. If we make them explicit in the schema, the question arises as to how we'd handle cases when they are missing?

It's okay if they're nullable. This is how our current reference data service handles it.

What are attributes in the schema? Are they just returned as an arbitrary JSON blob?

Currently we store triples in the reference data service. Each triple can have an attribute (which is an entity with name, triples, relations, etc.), value type, and value. It's sometimes important to filter the triples as well. For example we might be querying a text block and only care about specifically the MARKDOWN_CONTENT attribute.

Value types are super important for generic applications since we need to know the value type of the triple in order to know how to render it appropriately.

---

type in the Relation schema should also point to an entity instead of returning a String, since we'll want to be able to query for things like the name, it's schema, etc. similarly to the from and to entities.

[baiirun]

For the initial implementation we'll also need schemas exposed for governance-related things like members/editors, proposals, versions. Not sure if this proposal is meant to cover those.

[cvauclair]

@baiirun

What are attributes in the schema? Are they just returned as an arbitrary JSON blob?

Yes! With the exception that there are no nested JSON objects. It's basically just the contents of the Neo4j node.

It's sometimes important to filter the triples as well.

We can easily add filters to the attributes field.

Value types are super important for generic applications since we need to know the value type of the triple in order to know how to render it appropriately.

The Attributes type can definitely be re-worked to accommodate that use case! Although it would also be possible to have typed gql entities for that, e.g.:

query {
  # query an entity of type "LgKenoh2EfWrvqJqN6A7Ci" (attribute type) with id "gEfvT3cW16tyPmFEGA9bp"
  _LgKenoh2EfWrvqJqN6A7Ci(id: "gEfvT3cW16tyPmFEGA9bp") {
    name
    valueType
  }
}

or alternatively

query {
  # query an entity with id "gEfvT3cW16tyPmFEGA9bp" and match on type "LgKenoh2EfWrvqJqN6A7Ci" (attribute type)
  entity(id: "gEfvT3cW16tyPmFEGA9bp") {
    ...on _LgKenoh2EfWrvqJqN6A7Ci {
      name
      valueType
    }
  }
}

type in the Relation schema should also point to an entity instead of returning a String, since we'll want to be able to query for things like the name, it's schema, etc. similarly to the from and to entities.

For the initial implementation we'll also need schemas exposed for governance-related things like members/editors, proposals, versions. Not sure if this proposal is meant to cover those.

Yep already working on that!

[baiirun]

Correct me if I'm wrong, but in your examples it looks like you query the value type defined on the Attribute right? Not the value type for the op that creates the triple? Not all attributes have value types set and not all triples conform to an attribute's value type.

For a generalized knowledge graph browser like Geo, we need to be able to granularly render each triple independently from if the Attribute has a defined value type or not.

With your examples my understanding is that I couldn't write this query from the reference data service with the attributes approach. This query is fetching an entity's triples and returning a subset of the triples where the value type of the stored triples is TEXT. Not triples where the value type of the attribute is TEXT.

query {
    entity(id: "gEfvT3cW16tyPmFEGA9bp") {
        triples(filter: { valueType: { equalTo: 'TEXT' } }) {
            value
            valueType
            attribute {
                name
            } 
        }
    }
}