Add conditional validation in meta-schema

[fbdtemme]

Add conditional validation for all fields depending on the parameter type to the "meta-schema added in #133.

Additionally this also checks for unevaluated properties to catch typos and wrong properties using the unevaluatedProperties feature. This requires the schema spec to be bumped to version 2019-09.

The changes were tested on all nf-core pipelines:

Testing on all nf-core pipelines

Get the pipeline schema files:

wget https://nf-co.re/pipelines.json
cat pipelines.json | jq ".remote_workflows [].name" -r | xargs -I{} wget "https://raw.githubusercontent.com/nf-core/{}/master/nextflow_schema.json" -O {}_schema.json

Run the validation:

{ for f in *schema.json; do printf "# $f\n\n\`\`\`\n"; check-jsonschema --schemafile ../parameters_meta_schema.json $f; printf "\n\`\`\`\n\n"; done } | tee run.txt

Full output:

airrflow_schema.json

Schema validation errors were encountered.
  airrflow_schema.json::$.definitions.clonal_analysis_options.properties.clonal_threshold: Unevaluated properties are not allowed ('default', 'description', 'fa_icon', 'type' were unexpected)
  airrflow_schema.json::$.definitions.clonal_analysis_options.properties.clonal_threshold.type: ['string', 'number'] is not of type 'string'
  airrflow_schema.json::$.definitions.clonal_analysis_options.properties.clonal_threshold.type: ['string', 'number'] is not one of ['string', 'boolean', 'integer', 'number']

ampliseq_schema.json

Schema validation errors were encountered.
  ampliseq_schema.json::$.definitions.main_arguments.properties.input: Unevaluated properties are not allowed ('mimetype', 'schema' were unexpected)
  ampliseq_schema.json::$.definitions.main_arguments.properties.input_fasta: Unevaluated properties are not allowed ('mimetype' was unexpected)
  ampliseq_schema.json::$.definitions.main_arguments.properties.input_folder: Unevaluated properties are not allowed ('mimetype' was unexpected)

atacseq_schema.json

ok -- validation done

bacass_schema.json

Schema validation errors were encountered.
  bacass_schema.json::$.definitions.input_output_options.properties.input: Unevaluated properties are not allowed ('exist', 'mimetype', 'schema' were unexpected)
  bacass_schema.json::$.definitions.qc_and_trim.properties.save_trimmed_fail: Unevaluated properties are not allowed ('enum' was unexpected)

bactmap_schema.json

ok -- validation done

bamtofastq_schema.json

Schema validation errors were encountered.
  bamtofastq_schema.json::$.definitions.generic_options.properties.multiqc_logo: Unevaluated properties are not allowed ('exists' was unexpected)
  bamtofastq_schema.json::$.definitions.generic_options.properties.multiqc_methods_description: Unevaluated properties are not allowed ('exists' was unexpected)

cageseq_schema.json

Schema validation errors were encountered.
  cageseq_schema.json::$.definitions.alignment_options.properties.min_aln_length: Unevaluated properties are not allowed ('default' was unexpected)
  cageseq_schema.json::$.definitions.alignment_options.properties.min_aln_length.default: '15' is not of type 'integer', 'null'

callingcards_schema.json

ok -- validation done

chipseq_schema.json

ok -- validation done

circdna_schema.json

Schema validation errors were encountered.
  circdna_schema.json::$.definitions.amplicon_architect_options.properties.aa_data_repo: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circdna_schema.json::$.definitions.amplicon_architect_options.properties.aa_cngain: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circdna_schema.json::$.definitions.amplicon_architect_options.properties.mosek_license_dir: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circdna_schema.json::$.definitions.amplicon_architect_options.properties.reference_build: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circdna_schema.json::$.definitions.amplicon_architect_options.properties.cnvkit_cnn: Unevaluated properties are not allowed ('mimetype' was unexpected)

circrna_schema.json

Schema validation errors were encountered.
  circrna_schema.json::$.definitions.input_output_options.properties.input: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circrna_schema.json::$.definitions.input_output_options.properties.phenotype: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circrna_schema.json::$.definitions.pipeline_options.properties.module: Unevaluated properties are not allowed ('mandatory' was unexpected)
  circrna_schema.json::$.definitions.reference_genome_options.properties.gtf: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circrna_schema.json::$.definitions.reference_genome_options.properties.mature: Unevaluated properties are not allowed ('mimetype' was unexpected)
  circrna_schema.json::$.definitions.generic_options.properties.email: Unevaluated properties are not allowed ('mimetype' was unexpected)

clipseq_schema.json

ok -- validation done

coproid_schema.json

ok -- validation done

createpanelrefs_schema.json

ok -- validation done

createtaxdb_schema.json

ok -- validation done

crisprseq_schema.json

ok -- validation done

crisprvar_schema.json

Several files failed to parse.
  Failed to parse crisprvar_schema.json

cutandrun_schema.json

ok -- validation done

ddamsproteomics_schema.json

Several files failed to parse.
  Failed to parse ddamsproteomics_schema.json

deepvariant_schema.json

Several files failed to parse.
  Failed to parse deepvariant_schema.json

demultiplex_schema.json

ok -- validation done

denovohybrid_schema.json

Several files failed to parse.
  Failed to parse denovohybrid_schema.json

detaxizer_schema.json

ok -- validation done

diaproteomics_schema.json

ok -- validation done

differentialabundance_schema.json

Schema validation errors were encountered.
  differentialabundance_schema.json::$.definitions.features_options.properties.features: Unevaluated properties are not allowed ('mimetype' was unexpected)

dualrnaseq_schema.json

Schema validation errors were encountered.
  dualrnaseq_schema.json::$.definitions.star_general.properties.limitBAMsortRAM: Unevaluated properties are not allowed ('default' was unexpected)
  dualrnaseq_schema.json::$.definitions.star_general.properties.limitBAMsortRAM.default: '0' is not of type 'integer', 'null'

eager_schema.json

Schema validation errors were encountered.
  eager_schema.json::$.definitions.mapping.properties.mapper: Unevaluated properties are not allowed ('title' was unexpected)

epitopeprediction_schema.json

ok -- validation done

exoseq_schema.json

Several files failed to parse.
  Failed to parse exoseq_schema.json

fastquorum_schema.json

ok -- validation done

fetchngs_schema.json

ok -- validation done

funcscan_schema.json

ok -- validation done

genomeannotator_schema.json

Several files failed to parse.
  Failed to parse genomeannotator_schema.json

genomeassembler_schema.json

ok -- validation done

genomeskim_schema.json

ok -- validation done

gwas_schema.json

ok -- validation done

hgtseq_schema.json

ok -- validation done

hicar_schema.json

ok -- validation done

hic_schema.json

Several files failed to parse.
  Failed to parse hic_schema.json

hlatyping_schema.json

ok -- validation done

imcyto_schema.json

Several files failed to parse.
  Failed to parse imcyto_schema.json

isoseq_schema.json

ok -- validation done

kmermaid_schema.json

Several files failed to parse.
  Failed to parse kmermaid_schema.json

liverctanalysis_schema.json

ok -- validation done

lncpipe_schema.json

Several files failed to parse.
  Failed to parse lncpipe_schema.json

mag_schema.json

Schema validation errors were encountered.
  mag_schema.json::$.definitions.quality_control_for_short_reads_options.properties.fastp_qualified_quality: Unevaluated properties are not allowed ('help' was unexpected)
  mag_schema.json::$.definitions.quality_control_for_short_reads_options.properties.fastp_cut_mean_quality: Unevaluated properties are not allowed ('help' was unexpected)

marsseq_schema.json

ok -- validation done

mcmicro_schema.json

ok -- validation done

metaboigniter_schema.json

ok -- validation done

metapep_schema.json

ok -- validation done

metatdenovo_schema.json

ok -- validation done

methylseq_schema.json

Schema validation errors were encountered.
  methylseq_schema.json::$.definitions.qualimap_options.properties.bamqc_regions_file: Unevaluated properties are not allowed ('description', 'help_text' were unexpected)
  methylseq_schema.json::$.definitions.qualimap_options.properties.bamqc_regions_file: 'type' is a required property

mhcquant_schema.json

Schema validation errors were encountered.
  mhcquant_schema.json::$.definitions.mass_spectrometry_data_processing.properties.annotate_ions: Unevaluated properties are not allowed ('default' was unexpected)
  mhcquant_schema.json::$.definitions.mass_spectrometry_data_processing.properties.annotate_ions.default: 'false' is not of type 'boolean', 'null'

mnaseseq_schema.json

Several files failed to parse.
  Failed to parse mnaseseq_schema.json

molkart_schema.json

ok -- validation done

multiplesequencealign_schema.json

ok -- validation done

nanoseq_schema.json

Schema validation errors were encountered.
  nanoseq_schema.json::$.definitions.input_output_options.properties.protocol: Unevaluated properties are not allowed ('format', 'mimetype', 'schema' were unexpected)
  nanoseq_schema.json::$.definitions.input_output_options.properties.protocol.format: 'sample-type' is not one of ['file-path', 'directory-path', 'path', 'file-path-pattern']

nanostring_schema.json

ok -- validation done

nascent_schema.json

ok -- validation done

neutronstar_schema.json

Several files failed to parse.
  Failed to parse neutronstar_schema.json

pangenome_schema.json

Schema validation errors were encountered.
  pangenome_schema.json::$.definitions.wfmash_options.properties.wfmash_sparse_map: Unevaluated properties are not allowed ('default' was unexpected)
  pangenome_schema.json::$.definitions.wfmash_options.properties.wfmash_sparse_map.default: '1.0' is not of type 'number', 'null'

pathogensurveillance_schema.json

ok -- validation done

pgdb_schema.json

ok -- validation done

phageannotator_schema.json

ok -- validation done

phyloplace_schema.json

ok -- validation done

pixelator_schema.json

Schema validation errors were encountered.
  pixelator_schema.json::$.definitions.adapterqc_options: 'type' is a required property
  pixelator_schema.json::$.definitions.demux_options: 'type' is a required property
  pixelator_schema.json::$.definitions.collapse_options: 'type' is a required property
  pixelator_schema.json::$.definitions.collapse_options.properties.markers_ignore: Unevaluated properties are not allowed ('fa-icon' was unexpected)
  pixelator_schema.json::$.definitions.graph_options: 'type' is a required property
  pixelator_schema.json::$.definitions.annotate_options: 'type' is a required property
  pixelator_schema.json::$.definitions.analysis_options: 'type' is a required property
  pixelator_schema.json::$.definitions.report_options: 'type' is a required property
  pixelator_schema.json::$.definitions.global_config_options.properties.pixelator_container: Unevaluated properties are not allowed ('format' was unexpected)
  pixelator_schema.json::$.definitions.global_config_options.properties.pixelator_container.format: 'url' is not one of ['file-path', 'directory-path', 'path', 'file-path-pattern']

proteinfold_schema.json

Schema validation errors were encountered.
  proteinfold_schema.json::$.definitions.colabfold_dbs_and_parameters_path_options.properties.colabfold_alphafold2_params_tags: Unevaluated properties are not allowed ('description', 'fa_icon', 'type' were unexpected)
  proteinfold_schema.json::$.definitions.colabfold_dbs_and_parameters_path_options.properties.colabfold_alphafold2_params_tags.type: 'object' is not one of ['string', 'boolean', 'integer', 'number']

proteomicslfq_schema.json

ok -- validation done

quantms_schema.json

Schema validation errors were encountered.
  quantms_schema.json::$.definitions.protein_quantification_dda.properties.ratios: Unevaluated properties are not allowed ('default' was unexpected)
  quantms_schema.json::$.definitions.protein_quantification_dda.properties.ratios.default: 'false' is not of type 'boolean', 'null'
  quantms_schema.json::$.definitions.protein_quantification_dda.properties.normalize: Unevaluated properties are not allowed ('default' was unexpected)
  quantms_schema.json::$.definitions.protein_quantification_dda.properties.normalize.default: 'false' is not of type 'boolean', 'null'
  quantms_schema.json::$.definitions.protein_quantification_dda.properties.fix_peptides: Unevaluated properties are not allowed ('default' was unexpected)
  quantms_schema.json::$.definitions.protein_quantification_dda.properties.fix_peptides.default: 'false' is not of type 'boolean', 'null'

radseq_schema.json

ok -- validation done

rangeland_schema.json

Schema validation errors were encountered.
  rangeland_schema.json::$.description: '' is too short
  rangeland_schema.json::$.definitions.input_output_options.properties.input: Unevaluated properties are not allowed ('mimetype' was unexpected)

raredisease_schema.json

Schema validation errors were encountered.
  raredisease_schema.json::$.definitions.reference_file_options.properties.bait_padding: Unevaluated properties are not allowed ('pattern' was unexpected)
  raredisease_schema.json::$.definitions.reference_file_options.properties.vcfanno_resources: Unevaluated properties are not allowed ('exists' was unexpected)
  raredisease_schema.json::$.definitions.reference_file_options.properties.vcfanno_toml: Unevaluated properties are not allowed ('exists' was unexpected)
  raredisease_schema.json::$.definitions.reference_file_options.properties.vcfanno_lua: Unevaluated properties are not allowed ('exists' was unexpected)
  raredisease_schema.json::$.definitions.reference_file_options.properties.vep_cache: Unevaluated properties are not allowed ('exists' was unexpected)

readsimulator_schema.json

ok -- validation done

riboseq_schema.json

Schema validation errors were encountered.
  riboseq_schema.json::$.definitions.reference_genome_options.properties.riboviz_index: Unevaluated properties are not allowed ('exists', 'format' were unexpected)
  riboseq_schema.json::$.definitions.reference_genome_options.properties.riboviz_index.format: 'dir-path' is not one of ['file-path', 'directory-path', 'path', 'file-path-pattern']
  riboseq_schema.json::$.definitions.Indexing.properties.build_indices: Unevaluated properties are not allowed ('default' was unexpected)
  riboseq_schema.json::$.definitions.Indexing.properties.build_indices.default: 'true' is not of type 'boolean', 'null'
  riboseq_schema.json::$.definitions.Trimming_options.properties.adapter_sequence: 'type' is a required property
  riboseq_schema.json::$.definitions.Trimming_options.properties.adapter_sequence.default: 'AGATCGGAAGAGCACACGTCTGAA' is not of type 'boolean', 'null'
  riboseq_schema.json::$.definitions.Trimming_options.properties.adapter_sequence.default: 'AGATCGGAAGAGCACACGTCTGAA' is not of type 'integer', 'null'
  riboseq_schema.json::$.definitions.Trimming_options.properties.adapter_sequence.default: 'AGATCGGAAGAGCACACGTCTGAA' is not of type 'number', 'null'
  riboseq_schema.json::$.definitions.Trimming_options.properties.single_end: Unevaluated properties are not allowed ('meta' was unexpected)

rnadnavar_schema.json

ok -- validation done

rnafusion_schema.json

ok -- validation done

rnaseq_schema.json

ok -- validation done

rnasplice_schema.json

ok -- validation done

rnavar_schema.json

ok -- validation done

sammyseq_schema.json

ok -- validation done

sarek_schema.json

ok -- validation done

scflow_schema.json

ok -- validation done

scrnaseq_schema.json

ok -- validation done

slamseq_schema.json

Several files failed to parse.
  Failed to parse slamseq_schema.json

smartseq2_schema.json

Several files failed to parse.
  Failed to parse smartseq2_schema.json

smrnaseq_schema.json

ok -- validation done

spatialtranscriptomics_schema.json

ok -- validation done

spinningjenny_schema.json

ok -- validation done

ssds_schema.json

ok -- validation done

taxprofiler_schema.json

Schema validation errors were encountered.
  taxprofiler_schema.json::$.definitions.preprocessing_general_qc_options.properties.skip_preprocessing_qc: Unevaluated properties are not allowed ('help' was unexpected)

variantcatalogue_schema.json

ok -- validation done

vipr_schema.json

Several files failed to parse.
  Failed to parse vipr_schema.json

viralintegration_schema.json

Schema validation errors were encountered.
  viralintegration_schema.json::$.definitions.input_output_options.properties.remove_duplicates: Unevaluated properties are not allowed ('default' was unexpected)
  viralintegration_schema.json::$.definitions.input_output_options.properties.remove_duplicates.default: 'true' is not of type 'boolean', 'null'

viralrecon_schema.json

ok -- validation done

The conditional validation catches some extra issues:

• mimetype and or schema being set without "format": "file-path".
• Unrecognised keys: mandatory (should be passed in required), help (should be help_text)
• Default values passed as strings instead of the actual type ('1.0' instead of 1.0)

[nvnieuwk]

Hi the current validation library does not support draft 2019-09, but we're working on changing our support to the 2020-12 draft (See #141) for the progress on that

[fbdtemme]

Nice. This can definitely wait until that work is merged.

[nvnieuwk]

Version 2.0.0 has been released, but we have to decide on what this meta schema would look like with the whole community and the people from Seqera, I'd put this on hold for a while longer until this has been full decided

[ewels]

We are moving to https://github.com/nextflow-io/nf-schema/ - see discussion on Slack.

Closing this PR, please reopen at the above repo 👍🏻

[ewels]

Changed my mind. Leaving open here until I see the new PRs pop up on https://github.com/nextflow-io/nf-schema/ 😄

Just don't want them to get lost.

[nvnieuwk]

This has been moved to nextflow-io#2