Merge pull request #283 from togoid/main

release 2024-12-17
togoid · Dec 17, 2024 · d23fc88 · d23fc88
2 parents 963af94 + 8161fc8
commit d23fc88
Show file tree

Hide file tree

Showing 8 changed files with 1,663 additions and 6,894 deletions.
diff --git a/docs/help.md b/docs/help.md
@@ -1,5 +1,5 @@
 # TogoID ver. 2.0
-Datasets last updated: 2024-12-09
+Datasets last updated: 2024-12-17
 
 ## About
 - [TogoID](https://togoid.dbcls.jp/) is an ID conversion service implementing unique features with an intuitive web interface and an API for programmatic access. TogoID supports datasets from various biological categories such as gene, protein, chemical compound, pathway, disease, etc. TogoID users can perform exploratory multistep conversions to find a path among IDs. To guide the interpretation of biological meanings in the conversions, we crafted an [ontology](https://togoid.dbcls.jp/ontology) that defines the semantics of the dataset relations.
@@ -10,7 +10,7 @@ Datasets last updated: 2024-12-09
 ## Video tutorial
 - [How to use TogoID: an exploratory ID converter to bridge biological datasets](https://youtu.be/gXnvm6Fn4R8)
 
-## Statistics (as of 2024-12-09)
+## Statistics (as of 2024-12-17)
 - Number of target datasets 
     - 105 (from 73 databases)
 - For details on the target DBs and ID examples, please refer to the "DATASETS" tab. 

diff --git a/docs/help_ja.md b/docs/help_ja.md
@@ -1,5 +1,5 @@
 # TogoID ver. 2.0
-Datasets last updated: 2024-12-09
+Datasets last updated: 2024-12-17
 
 ## About
 - [TogoID](https://togoid.dbcls.jp/) は、直感的なインターフェースにより生命科学系データベース(DB)間のつながりを探索的に確認しながらID変換を行うことができるウェブアプリケーションです。同一の実体を指すID間の変換だけでなく、関連する別のカテゴリーのIDへの変換も可能です。また、直接リンクされていないDBのID間でも、他のDBを経由した変換を探索することができます。
@@ -10,7 +10,7 @@ Datasets last updated: 2024-12-09
 ## 動画マニュアル
 - [TogoIDを使って生命科学系データベースのさまざまなIDを探索的に変換する](https://youtu.be/gXnvm6Fn4R8)
 
-## 統計 (2024-12-09)
+## 統計 (2024-12-17)
 - 対象データセット数 
     - 105 (73 のデータベースに由来)
 - 対象DBの詳細やID例については、"DATASETS" タブ からご覧いただけます。 

diff --git a/docs/news.md b/docs/news.md
@@ -1,8 +1,5 @@
-# *NOTICE*
-Some of the features of TogoID will be temporarily unavailable due to our system maintenance from 09:00 on December 13, 2024, to 18:00 on December 16, 2024 (JST).  
-The *LABEL2ID* function will not be available, and the *label display features* for some datasets will also be affected.  
-The *ID-to-ID conversion function will remain available* as usual throughout the maintenance period.  
-We apologize for any incovenience caused.  
+# 2024-12-17
+- Weekly update has been completed.
 
 # 2024-12-09
 - Weekly update has been completed.

diff --git a/log/config-summary.tsv b/log/config-summary.tsv
@@ -72,7 +72,7 @@ hgnc-ncbigene	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00073	Gene	NC
 hgnc-omim_gene	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00154	Gene	OMIM gene	http://identifiers.org/mim/	tio:TIO_000001	is nearly equivalent to	is nearly equivalent to	tio:TIO_000001	is nearly equivalent to	is nearly equivalent to	Bimonthly	"awk -F \"\\t\" 'FNR>=2&&$32{gsub(\"\\\"\",\"\",$32); split($32,a,\"|\"); for(k in a)print substr($1,6) \"\\t\" a[k]}' $TOGOID_ROOT/input/hgnc/hgnc_complete_set.txt"
 hgnc-pubmed	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00179	Literature	PubMed	http://rdf.ncbi.nlm.nih.gov/pubmed/	tio:TIO_000032	has reference	has reference	tio:TIO_000033	is reference of	is reference of	Bimonthly	"awk -F \"\\t\" 'FNR>=2&&$27{gsub(\"\\\"\",\"\",$27); split($27,a,\"|\"); for(k in a)print substr($1,6) \"\\t\" a[k]}' $TOGOID_ROOT/input/hgnc/hgnc_complete_set.txt"
 hgnc-refseq_genomic	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00187	Gene	RefSeq genomic	http://identifiers.org/refseq/	tio:TIO_000001	is nearly equivalent to	is nearly equivalent to	tio:TIO_000001	is nearly equivalent to	is nearly equivalent to	Bimonthly	"awk -F \"\\t\" 'FNR>=2&&$24{gsub(\"\\\"\",\"\",$24); split($24,a,\"|\"); for(k in a){if(a[k]~/^NG_/)print substr($1,6) \"\\t\" a[k]}}' $TOGOID_ROOT/input/hgnc/hgnc_complete_set.txt"
-hgnc-refseq_rna	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00187	Transcript	RefSeq RNA	http://identifiers.org/refseq/	tio:TIO_000035	gene is transcribed to transcript	is transcribed to	tio:TIO_000034	transcript is transcribed from gene	is transcribed from	Bimonthly	"awk -F \"\\t\" 'FNR>=2&&$24{gsub(\"\\\"\",\"\",$24); split($24,a,\"|\"); for(k in a){if(a[k]~/^[NX][MR]_/)print substr($1,6) \"\\t\" a[k]}}' $TOGOID_ROOT/input/hgnc/hgnc_complete_set.txt"
+hgnc-refseq_rna	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00187	Transcript	RefSeq RNA	http://identifiers.org/refseq/	tio:TIO_000035	gene is transcribed to transcript	is transcribed to	tio:TIO_000034	transcript is transcribed from gene	is transcribed from	Bimonthly	"awk -F \"\\t\" 'FNR>=2&&$24{gsub(\"\\\"\",\"\",$24); split($24,a,\"|\"); for(k in a){if(a[k]~/^[NX][MR]_/)print substr($1,6) \"\\t\" gensub(/\\.$/,\"\",\"g\",a[k])}}' $TOGOID_ROOT/input/hgnc/hgnc_complete_set.txt"
 hgnc-rgd	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00188	Gene	RGD	http://identifiers.org/rgd/	tio:TIO_000017	sequence is orthologous to sequence	is orthologous to	tio:TIO_000017	sequence is orthologous to sequence	is orthologous to	Bimonthly	"awk -F \"\\t\" 'FNR>=2&&$29{gsub(\"\\\"\",\"\",$29); split($29,a,\"|\"); for(k in a)print substr($1,6) \"\\t\" substr(a[k],5)}' $TOGOID_ROOT/input/hgnc/hgnc_complete_set.txt"
 hgnc-uniprot	nbdc01774	Gene	HGNC	http://identifiers.org/hgnc/	nbdc00221	Protein	UniProt	http://purl.uniprot.org/uniprot/	tio:TIO_000025	gene has protein as product	has gene product	tio:TIO_000024	protein is product of gene	is product of gene	Bimonthly	"awk -F \"\\t\" 'FNR>=2&&$26{gsub(\"\\\"\",\"\",$26); split($26,a,\"|\"); for(k in a)print substr($1,6) \"\\t\" a[k]}' $TOGOID_ROOT/input/hgnc/hgnc_complete_set.txt"
 hmdb-chebi	nbdc00909	Compound	HMDB	http://identifiers.org/hmdb/	nbdc00027	Compound	ChEBI compound	http://purl.obolibrary.org/obo/CHEBI_	tio:TIO_000001	is nearly equivalent to	is nearly equivalent to	tio:TIO_000001	is nearly equivalent to	is nearly equivalent to	Triennial	"awk -F \"\\t\" '$2==\"chebi_id\"{print $1 \"\\t\" $3}' $TOGOID_ROOT/input/hmdb/hmdb_metabolites.tsv"

diff --git a/log/error.log b/log/error.log
@@ -1,13 +1,72 @@
-Error: download_file(input/glytoucan, https://glycosmos.org/download/glycosmos_ggdbs_pubmed.csv): Command failed with status (8): [wget --quiet --recursive --no-parent --no-...]
 Error: check_remote_file_time(input/homologene/homologene.data, https://ftp.ncbi.nlm.nih.gov/pub/HomoloGene/current/homologene.data): no time information in ""
 Error: Remote file is empty
+# Error: output/tsv/affy_probeset-ncbigene.tsv new file size per old 248 / 314378 = 0.000788859271323057 < 0.5
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <html><head>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <title>503 Service Unavailable</title>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML </head><body>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <h1>Service Unavailable</h1>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <p>The server is temporarily unable to service your
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to be malformed request due to maintenance downtime or capacity
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML problems. Please try again later.</p>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML </body></html>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <html><head>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <title>503 Service Unavailable</title>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML </head><body>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <h1>Service Unavailable</h1>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML <p>The server is temporarily unable to service your
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to be malformed request due to maintenance downtime or capacity
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML problems. Please try again later.</p>
+# Error: output/tsv/affy_probeset-ncbigene.tsv seems to contain HTML </body></html>
+# Error: output/tsv/assembly_insdc-bioproject.tsv new file size per old 0 / 58955319 = 0.0 < 0.5
+# Error: Failed to create output/tsv/assembly_insdc-bioproject.tsv or created file was empty
+# Error: output/tsv/assembly_insdc-biosample.tsv new file size per old 0 / 60555966 = 0.0 < 0.5
+# Error: Failed to create output/tsv/assembly_insdc-biosample.tsv or created file was empty
+# Error: output/tsv/assembly_insdc-insdc_master.tsv new file size per old 0 / 63716447 = 0.0 < 0.5
+# Error: Failed to create output/tsv/assembly_insdc-insdc_master.tsv or created file was empty
+# Error: output/tsv/bioproject-biosample.tsv new file size per old 0 / 63710867 = 0.0 < 0.5
+# Error: Failed to create output/tsv/bioproject-biosample.tsv or created file was empty
+# Error: output/tsv/bioproject_umbrella-bioproject.tsv new file size per old 0 / 2475955 = 0.0 < 0.5
+# Error: Failed to create output/tsv/bioproject_umbrella-bioproject.tsv or created file was empty
 # Error: output/tsv/chembl_target-ensembl_gene.tsv new file size per old 0 / 61845 = 0.0 < 0.5
 # Error: Failed to create output/tsv/chembl_target-ensembl_gene.tsv or created file was empty
+# Error: output/tsv/ensembl_gene-ensembl_protein.tsv new file size per old 0 / 493740466 = 0.0 < 0.5
+# Error: Failed to create output/tsv/ensembl_gene-ensembl_protein.tsv or created file was empty
+# Error: output/tsv/ensembl_gene-ensembl_transcript.tsv new file size per old 0 / 606499011 = 0.0 < 0.5
+# Error: Failed to create output/tsv/ensembl_gene-ensembl_transcript.tsv or created file was empty
+# Error: output/tsv/ensembl_gene-ncbigene.tsv new file size per old 0 / 425488063 = 0.0 < 0.5
+# Error: Failed to create output/tsv/ensembl_gene-ncbigene.tsv or created file was empty
+# Error: output/tsv/ensembl_gene-uniprot.tsv new file size per old 0 / 159598297 = 0.0 < 0.5
+# Error: Failed to create output/tsv/ensembl_gene-uniprot.tsv or created file was empty
+# Error: output/tsv/ensembl_protein-ensembl_transcript.tsv new file size per old 0 / 493651516 = 0.0 < 0.5
+# Error: Failed to create output/tsv/ensembl_protein-ensembl_transcript.tsv or created file was empty
+# Error: output/tsv/ensembl_transcript-go.tsv new file size per old 0 / 1491765002 = 0.0 < 0.5
+# Error: Failed to create output/tsv/ensembl_transcript-go.tsv or created file was empty
+# Error: output/tsv/ensembl_transcript-refseq_rna.tsv new file size per old 0 / 1283309099 = 0.0 < 0.5
+# Error: Failed to create output/tsv/ensembl_transcript-refseq_rna.tsv or created file was empty
+# Error: output/tsv/gea-bioproject.tsv new file size per old 0 / 3032 = 0.0 < 0.5
+# Error: Failed to create output/tsv/gea-bioproject.tsv or created file was empty
+# Error: output/tsv/gea-biosample.tsv new file size per old 0 / 121104 = 0.0 < 0.5
+# Error: Failed to create output/tsv/gea-biosample.tsv or created file was empty
 Error: HTTP Error 500: Internal Server Error: https://api.alpha.glycosmos.org/partialmatch?wurcs=WURCS%3D2.0%2F4%2C11%2C10%2F%5Ba2122h-1b_1-5%5D%5Ba2112h-1b_1-5%5D%5Ba2122h-1b_1-5_2%2ANCC%2F3%3DO%5D%5Ba1221m-1a_1-5%5D%2F1-2-3-2-3-2-3-4-2-3-2%2Fa4-b1_b3-c1_c4-d1_d3-e1_e4-f1_f3-g1_g3-h1_g4-i1_i3-j1_j4-k1&rootnode=true G64227KZ
 # Error: output/tsv/glycomotif-glytoucan.tsv new file size per old 0 / 2804382 = 0.0 < 0.5
 # Error: Failed to create output/tsv/glycomotif-glytoucan.tsv or created file was empty
-# Error: output/tsv/glytoucan-doid.tsv new file size per old 1538 / 4376 = 0.35146252285191953 < 0.5
+# Error: output/tsv/glytoucan-doid.tsv new file size per old 0 / 4376 = 0.0 < 0.5
+# Error: Failed to create output/tsv/glytoucan-doid.tsv or created file was empty
 # Error: output/tsv/hgnc-ccds.tsv new file size per old 252491 / 544502 = 0.46370995882476096 < 0.5
+# Error: output/tsv/insdc-bioproject.tsv new file size per old 0 / 1191557755 = 0.0 < 0.5
+# Error: Failed to create output/tsv/insdc-bioproject.tsv or created file was empty
+# Error: output/tsv/insdc-biosample.tsv new file size per old 0 / 4269980335 = 0.0 < 0.5
+# Error: Failed to create output/tsv/insdc-biosample.tsv or created file was empty
+# Error: output/tsv/insdc_master-bioproject.tsv new file size per old 0 / 41801168 = 0.0 < 0.5
+# Error: Failed to create output/tsv/insdc_master-bioproject.tsv or created file was empty
+# Error: output/tsv/insdc_master-biosample.tsv new file size per old 0 / 43765492 = 0.0 < 0.5
+# Error: Failed to create output/tsv/insdc_master-biosample.tsv or created file was empty
+# Error: output/tsv/jga_study-jga_dataset.tsv new file size per old 0 / 4378 = 0.0 < 0.5
+# Error: Failed to create output/tsv/jga_study-jga_dataset.tsv or created file was empty
+# Error: output/tsv/jga_study-nbdc_human_db.tsv new file size per old 0 / 5795 = 0.0 < 0.5
+# Error: Failed to create output/tsv/jga_study-nbdc_human_db.tsv or created file was empty
+# Error: output/tsv/jga_study-pubmed.tsv new file size per old 0 / 2597 = 0.0 < 0.5
+# Error: Failed to create output/tsv/jga_study-pubmed.tsv or created file was empty
 # Error: output/tsv/lipidmaps-chebi.tsv new file size per old 148 / 253350 = 0.000584172093941188 < 0.5
 # Error: output/tsv/lipidmaps-chebi.tsv seems to contain HTML <html><head>
 # Error: output/tsv/lipidmaps-chebi.tsv seems to contain HTML <title>403 Forbidden</title>
@@ -63,46 +122,4 @@ Error: HTTP Error 500: Internal Server Error: https://api.alpha.glycosmos.org/pa
 # Error: Failed to create output/tsv/mondo-omim_phenotype.tsv or created file was empty
 # Error: output/tsv/mondo-orphanet_phenotype.tsv new file size per old 0 / 146923 = 0.0 < 0.5
 # Error: Failed to create output/tsv/mondo-orphanet_phenotype.tsv or created file was empty
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv new file size per old 291 / 385314 = 0.0007552282034911786 < 0.5
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <html><head>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <title>408 Request Timeout</title>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML </head><body>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <h1>Request Timeout</h1>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <p>Server timeout waiting for the HTTP request from the client.</p>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <p>Additionally	 a 502 Bad Gateway
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML error was encountered while trying to use an ErrorDocument to handle the request.</p>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML </body></html>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <html><head>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <title>408 Request Timeout</title>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML </head><body>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <h1>Request Timeout</h1>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <p>Server timeout waiting for the HTTP request from the client.</p>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML <p>Additionally	 a 502 Bad Gateway
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML error was encountered while trying to use an ErrorDocument to handle the request.</p>
-# Error: output/tsv/ncit_disease-ncit_tissue.tsv seems to contain HTML </body></html>
-# Error: output/tsv/togovar-ensembl_transcript.tsv new file size per old 0 / 8069270901 = 0.0 < 0.5
-# Error: Failed to create output/tsv/togovar-ensembl_transcript.tsv or created file was empty
-# Error: output/tsv/togovar-ncbigene.tsv new file size per old 0 / 1097267175 = 0.0 < 0.5
-# Error: Failed to create output/tsv/togovar-ncbigene.tsv or created file was empty
-# Error: output/tsv/togovar-pubmed.tsv new file size per old 0 / 14408635 = 0.0 < 0.5
-# Error: Failed to create output/tsv/togovar-pubmed.tsv or created file was empty
-# Error: output/tsv/togovar-refseq_rna.tsv new file size per old 0 / 4180808277 = 0.0 < 0.5
-# Error: Failed to create output/tsv/togovar-refseq_rna.tsv or created file was empty
-# Error: output/tsv/uberon-ncit_tissue.tsv new file size per old 288 / 38722 = 0.007437632353700738 < 0.5
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <html><head>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <title>502 Proxy Error</title>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML </head><body>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <h1>Proxy Error</h1>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <p>The proxy server received an invalid
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML response from an upstream server.<br />
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML The proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML </body></html>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <html><head>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <title>502 Proxy Error</title>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML </head><body>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <h1>Proxy Error</h1>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML <p>The proxy server received an invalid
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML response from an upstream server.<br />
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML The proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>
-# Error: output/tsv/uberon-ncit_tissue.tsv seems to contain HTML </body></html>
 # Error: output/tsv/wikipathways-uniprot.tsv new file size per old 60240 / 470542 = 0.12802257821830995 < 0.5