💾 Archived View for thebird.nl › gn-gemtext-threads › issues › dump-genewiki-metadata.gmi captured on 2023-04-19 at 22:52:37. Gemini links have been rewritten to link to archived content

View Raw

More Information

⬅️ Previous capture (2023-03-20)

🚧 View Differences

-=-=-=-=-=-=-

Dump GeneWiki Metadata

Tags

• assigned: bonfacem
• type: feature
• keywords: metadata, RDF

Dump the tables:

• GeneRIF
• GeneRIF_BASIC

Resources

RDF Schema 1.1

RDF/S quick walk through

DCMI Metadata Terms

UNIPROT sparql examples

Checking for duplicates

ag "Observational study of gene-disease association" dump.ttl --pager='less -R'
ag "gn:symbol" | sort | less
ag "gn:anonSymbol" | sort | less

Issues

• Some entries in the GeneRIF table don't have any entries in the GeneRIF_BASIC table:

SELECT * FROM GeneRIF LEFT JOIN GeneRIF_BASIC USING (symbol)
LEFT JOIN GeneRIFXRef ON GeneRIFXRef.GeneRIFId = GeneRIF.Id
LEFT JOIN GeneCategory ON GeneRIFXRef.GeneCategoryId = GeneCategory.Id
WHERE GeneRIF.display > 0 AND GeneRIF.VersionId = 0
AND GeneRIF_BASIC.GeneId IS NULL\G

• Missing data: some GeneIds stored in GN are not in GeneInfo. These can be conceptualised as anonymous genewiki entries. One example is the symbol "Mul1" with the GeneId 68350. This example exists in GN with different GeneId's. Example query:

SELECT * FROM GeneInfo WHERE GeneId = 68350\G 

• NEWENTRY: We have many genes with the "NEWENTRY". In GN1, these are represented in one very big page:

https://gn1.genenetwork.org/webqtl/main.py?FormID=geneWiki&symbol=NEWENTRY

To query these entries:

SELECT * FROM GeneRIF_BASIC WHERE symbol = 'NEWENTRY'\G

• Broken UTF-8 character sets that rapper errored out on and that had to be manually fixed. Here's a list:

'(("\x28" . "")
  ("\x29" . "")
  ("\xa0" . " ")
  ("â\x81„" . "/")
  ("â€\x9d" . #\")
  ("’" . #\')
  ("\x02" . "")
  ("\x01" . "")
  ("β" . "β")
  ("α-Â\xad" . "α")
  ("Â\xad" . "")
  ("α" . "α")
  ("–" . "-"))

• In the GeneRIF_BASIC table, there are 14,313 rows with an empty symbol:

SELECT COUNT(*) FROM GeneRIF_BASIC WHERE symbol = '';

• The are comments with the same values but different GeneIds. Example:

https://gn1.genenetwork.org/webqtl/main.py?FormID=geneWiki&symbol=A2m

• closed