TogoDX server

TogoDX server is the backend server for TogoDX.

TogoDX (Togo Data eXplorer) is a framework to explore various databases integrated using a knowledge graph in the life sciences. It integrates many datasets originating from different databases based on the compatibility of IDs that indicate the same thing and ID relationships that organize the connections between different data. TogoDX aims to provide a new mechanism for flexibly extracting data useful for data science by narrowing down the data by various attributes of each data set.

Prerequisites

Ruby 3.0.2
Ruby on Rails 6.1.3
Either of the following
- SQLite 3.8+
- PostgreSQL 14+

System dependencies

TogoDX

Setup

Before installing gems

Use SQLite3

Comment out gem 'pg' in Gemfile.

Use PostgreSQL instead of SQLite3

Comment out gem 'sqlite3' in Gemfile.
Create .env and fill values

TOGODX_SERVER_DATABASE_HOST=
TOGODX_SERVER_DATABASE_PORT=
TOGODX_SERVER_DATABASE_USER=
TOGODX_SERVER_DATABASE_PASSWORD=

Variable	Default
TOGODX_SERVER_DATABASE_HOST	localhost
TOGODX_SERVER_DATABASE_PORT	5432
TOGODX_SERVER_DATABASE_USER	togodx

Note: The TOGODX_SERVER_DATABASE_USER must have CREATEDB privilege

Install gems

$ bundle install

Initialize application

$ ./bin/togodx init config
$ ./bin/togodx init bin

Database initialization

Run the following commands to create and setup the database.

$ bundle exec rake db:create
$ bundle exec rake db:migrate

Attributes config

Define attributes handled by the database in config/attributes.yml.

See example in example/attributes.yml

Dataset preparation

The supported formats of the data import script are JSON, TSV or CSV. For JSON, the data must be an array of objects. For TSV or CSV, the header line is required.

Data structure

Classification

Name	Type	Description
id	`String`	An identifier of the attribute value
label	`String`	A readable label for the identifier
parent	`String`	An identifier of the parent node
leaf	`Boolean`	Set `true` if this node is ID, or `false` if this node is category.

See example in example/gene_chromosome_ensembl.json

Distribution

Name	Type	Description
id	`String`	An identifier of the attribute value
label	`String`	A readable label for the identifier
value	`Number`	A value for the identifier
binId	`Number`	An identifier of the bin
binLabel	`String`	A readable label for the bin

See example in example/protein_molecular_mass_uniprot.json

Relation

This file should be prepared for each pair of dataset names

Execute rails runner 'p Relation.datasets' to get pair of dataset names.

Name	Type	Description
source	`String`	Identifier of source
target	`String`	Identifier of target

Import datasets

Attribute

$ ./bin/togodx attribute import

Relation

$ ./bin/togodx relation import

Use docker-compose

Make symbolic link

For development

$ ln -s docker/docker-compose.dev.yml docker-compose.yml

For production

$ ln -s docker/docker-compose.prod.yml docker-compose.yml

Create .env file

# Docker
NGINX_PORT=80 # for production
APP_PORT=3000 # for development

# Rails
TOGODX_SERVER_DATABASE_HOST=db
TOGODX_SERVER_DATABASE_USER=togodx
TOGODX_SERVER_DATABASE_PASSWORD=changeme

# PostgreSQL
POSTGRES_USER=togodx
POSTGRES_PASSWORD=changeme

Initialization and import data

$ docker-compose run --rm app bundle install
$ docker-compose run --rm app bin/togodx init config
$ docker-compose run --rm app bin/togodx init bin
$ docker-compose run --rm app rails db:create
$ docker-compose run --rm app rails db:migrate
$ docker-compose run --rm app bin/togodx attribute import
$ docker-compose run --rm app bin/togodx relation import

Start server

$ docker-compose up -d

Then visit to http://localhost:<APP_PORT>/breakdown/gene_chromosome_ensembl?pretty

API

`GET` `POST` /breakdown/{attribute}

Path parameter

parameter	type	required	description
attribute	String	Yes	attribute name

Query parameter `GET` / Request body `POST`

parameter	type	required	description
node	String	No	if not set, it is assumed to be the root node
hierarchy	Flag	No	if set, return parents and children of the node
order	String	No	`id_asc`\|`id_desc`\|`numerical_asc`\|`numerical_desc`\|`alphabetical_asc`\|`alphabetical_desc`

Remember to set request header Content-Type: application/json for POST

Response

hierarchy = false

[
  {
    "node": "node_1",
    "label": "label 1",
    "count": 10,
    "tip": true
  },
  {
    "node": "node_2",
    "label": "label 2",
    "count": 20,
    "tip": false
  },
  .
  .
  .
]

hierarchy = true

{
  "self": {
    "node": "node_3",
    "label": "label 3",
    "count": 30,
    "tip": false
  },
  "parents": [
    {
      "node": "node_4",
      "label": "label 4",
      "count": 40,
      "tip": false
    },
    {
      "node": "node_5",
      "label": "label 5",
      "count": 50,
      "tip": false
    }
  ],
  "children": [
    {
      "node": "node_1",
      "label": "label 1",
      "count": 10,
      "tip": true
    },
    {
      "node": "node_2",
      "label": "label 2",
      "count": 20,
      "tip": false
    },
    .
    .
    .
  ]
}

Suggest

`GET` `POST` /suggest/{attribute}

Path parameter

parameter	type	required	description
attribute	String	Yes	attribute name

Query parameter `GET` / Request body `POST`

parameter	type	required	description
term	String	Yes	query string (at least 3 characters)

Remember to set request header Content-Type: application/json for POST

Response

{
  "results": [
    {
      "node": "ndoe_1",
      "label": "label 1"
    },
    {
      "node": "ndoe_2",
      "label": "label 2"
    },
    .
    .
    .
  ],
  "total": 100
}

Locate

`POST` /locate/{attribute}

Path parameter

parameter	type	required	description
attribute	String	Yes	attribute name

Request body

parameter	type	required	description
dataset	String	Yes	target dataset name
queries	Array<String>	Yes	list of nodes
node	String	No	if not set, it is assumed to be the root node

Remember to set request header Content-Type: application/json

Response

[
  {
    "node": "node_1",
    "label": "label 1",
    "count": 100,
    "mapped": 0,
    "pvalue": null
  },
  {
    "node": "node_2",
    "label": "label 2",
    "count": 200,
    "mapped": 1,
    "pvalue": 0.12345
  },
  .
  .
  .
]

Aggregate

`POST` /aggregate

Request body

parameter	type	required	description
dataset	String	Yes	target dataset name
filters	Array<Object>	Yes	list of filters (see below)

Object structure of a filter

{
  "attribute": "attribute_1",
  "nodes": [
    "node_1",
    "node_1",
    .
    .
    .
  ]
}

Remember to set request header Content-Type: application/json

Response

[
  "id_1",
  "id_2",
  .
  .
  .
]

Dataframe

`POST` /dataframe

Request body

parameter	type	required	description
dataset	String	Yes	target dataset name
filters	Array<Object>	Yes	list of filters (see below)
annotations	Array<Object>	No	list of annotations (see below)
queries	Array<String>	Yes	list of queries

Object structure of a filter

{
  "attribute": "attribute_1",
  "nodes": [
    "node_1",
    "node_2",
    .
    .
    .
  ]
}

Object structure of a annotation

[
  {
    "attribute": "attribute_1"
  },
  {
    "attribute": "attribute_1"
  },
  .
  .
  .
]

Remember to set request header Content-Type: application/json

Response

[
  {
    "index": {
      "dataset": "dataset_1",
      "entry": "node_1",
      "label": "label 1"
    },
    "attributes": [
      {
        "id": "attribute_1",
        "items": [
          {
            "dataset": "dataset_2",
            "entry": "node_1",
            "node": "node_2",
            "label": "label 2"
          }
        ]
      },
      .
      .
      .
    ]
  },
  .
  .
  .
]

Name		Name	Last commit message	Last commit date
Latest commit History 200 Commits
app		app
bin		bin
config		config
db/migrate		db/migrate
docker		docker
example		example
lib/tasks		lib/tasks
log		log
public		public
spec		spec
storage		storage
templates		templates
tmp		tmp
vendor		vendor
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
.rspec		.rspec
Gemfile		Gemfile
Gemfile.lock		Gemfile.lock
README.md		README.md
Rakefile		Rakefile
config.ru		config.ru

togodx/togodx-server

Folders and files

Latest commit

History

Repository files navigation

TogoDX server

Prerequisites

System dependencies

Setup

Before installing gems

Use SQLite3

Use PostgreSQL instead of SQLite3

Install gems

Initialize application

Database initialization

Attributes config

Dataset preparation

Data structure

Classification

Distribution

Relation

Import datasets

Attribute

Relation

Use docker-compose

API

GET POST /breakdown/{attribute}

Path parameter

Query parameter GET / Request body POST

Response

Suggest

GET POST /suggest/{attribute}

Path parameter

Query parameter GET / Request body POST

Response

Locate

POST /locate/{attribute}

Path parameter

Request body

Response

Aggregate

POST /aggregate

Request body

Response

Dataframe

POST /dataframe

Request body

Response

About

Resources

Stars

Watchers

Forks

Releases

Packages 0

Contributors 5

Languages

`GET` `POST` /breakdown/{attribute}

Query parameter `GET` / Request body `POST`

`GET` `POST` /suggest/{attribute}

Query parameter `GET` / Request body `POST`

`POST` /locate/{attribute}

`POST` /aggregate

`POST` /dataframe

Packages