Spatial Simulators

Assessing the quality of spatial transcriptomics simulators

Repository: openproblems-bio/task_spatial_simulators

Description

Computational methods for spatially resolved transcriptomics (SRT) are frequently developed and assessed through data simulation. The effectiveness of these evaluations relies on the simulation methods’ ability to accurately reflect experimental data. However, a systematic evaluation framework for spatial simulators is lacking. Here, we present SpatialSimBench, a comprehensive evaluation framework that assesses 13 simulation methods using 10 distinct STR datasets.

The research goal of this benchmark is to systematically evaluate and compare the performance of various simulation methods for spatial transcriptomics (ST) data. It aims to address the lack of a comprehensive evaluation framework for spatial simulators and explore the feasibility of leveraging existing single-cell simulators for ST data. The experimental setup involves collecting public spatial transcriptomics datasets and corresponding scRNA-seq datasets. The spatial and scRNA-seq datasets can originate from different study but should consist of similar cell types from similar tissues.

Authors & contributors

name	roles
Xiaoqi Liang	author, maintainer
Yue Cao	author
Jean Yang	author
Robrecht Cannoodt	contributor
Sai Nirmayi Yasa	contributor

API

flowchart LR
  comp_process_datasets[/"Process Dataset"/]
  file_dataset_sc("Single-Cell Dataset")
  file_dataset_sp("Spatial Dataset")
  comp_metric[/"Metric"/]
  comp_control_method[/"Control Method"/]
  comp_method[/"Method"/]
  file_score("Score")
  file_simulated_dataset("Solution")
  comp_process_datasets-->file_dataset_sc
  comp_process_datasets-->file_dataset_sp
  file_dataset_sc---comp_metric
  file_dataset_sp---comp_metric
  file_dataset_sp---comp_control_method
  file_dataset_sp---comp_method
  comp_metric-->file_score
  comp_control_method-->file_simulated_dataset
  comp_method-->file_simulated_dataset
  file_simulated_dataset---comp_metric

Loading

Component type: Process Dataset

Preprocessing of spatial transcriptomics and single-cell transcriptomics datasets.

Arguments:

Name	Type	Description
--input_sc	file	NA.
--input_sp	file	NA.
--output_sc	file	(Output) An unprocessed single-cell dataset as output by a dataset loader. Default: $id/output_sc.h5ad.
--output_sp	file	(Output) An unprocessed spatial dataset as output by a dataset loader. Default: $id/output_sp.h5ad.
--dataset_id	string	NA.
--dataset_name	string	NA.
--dataset_url	string	(Optional) NA.
--dataset_reference	string	(Optional) NA.
--dataset_summary	string	NA.
--dataset_description	string	NA.
--dataset_organism	string	NA.
--dataset_assay_spatial	string	NA.
--dataset_assay_singlecell	string	NA.

File format: Single-Cell Dataset

An unprocessed single-cell dataset as output by a dataset loader.

Example file: resources_test/spatialsimbench_mobnew/dataset_sc.h5ad

Description:

This dataset contains raw counts and metadata as output by a dataset loader.

The format of this file is derived from the CELLxGENE schema v4.0.0.

File format: Spatial Dataset

An unprocessed spatial dataset as output by a dataset loader.

Example file: resources_test/spatialsimbench_mobnew/dataset_sp.h5ad

Description:

This dataset contains raw counts and metadata as output by a dataset loader.

The format of this file is derived from the CELLxGENE schema v4.0.0.

Component type: Metric

A metric.

Arguments:

Name	Type	Description
--input_spatial_dataset	file	An unprocessed spatial dataset as output by a dataset loader.
--input_singlecell_dataset	file	An unprocessed single-cell dataset as output by a dataset loader.
--input_simulated_dataset	file	The solution for the test data.
--output	file	(Output) File indicating the score of a metric.

Component type: Control Method

A control method.

Arguments:

Name	Type	Description
--input	file	(Optional) An unprocessed spatial dataset as output by a dataset loader.
--output	file	(Output) The solution for the test data.

Component type: Method

A method.

Arguments:

Name	Type	Description
--input	file	(Optional) An unprocessed spatial dataset as output by a dataset loader.
--base	string	(Optional) NA. Default: domain.
--base	string	(Optional) NA. Default: domain.
--output	file	(Output) The solution for the test data.

File format: Score

File indicating the score of a metric.

Example file: resources_test/spatialsimbench_mobnew/score.h5ad

File format: Solution

The solution for the test data

Example file: resources_test/spatialsimbench_mobnew/simulated_dataset_processed.h5ad