💾 Archived View for thebird.nl › gn-gemtext-threads › topics › next-gen-databases › design-doc.gmi captured on 2023-07-10 at 13:53:48. Gemini links have been rewritten to link to archived content

More Information

-=-=-=-=-=-=-

LMDB Phenotype/Genotype Store

lead: bonfacem
team: fred, alex
contact: pjotr

Please go through:

https://github.com/BonfaceKilz/gn-dataset-dump

https://github.com/genenetwork/dump-genenetwork-database

Tasks

Go through the above repository. We'll have a check in on Friday 23 to discuss this.
Write and package guile-lmdb. Use cl-lmdb as inspiration
Find out a more efficient way to store utf-8 strings
Integrate LMDB in GN:
Rewrite some statistical computations using LMDB+RDF
Integrate editing LMDB entries using Fred's new auth system

Actionable items (Week 1):

Bonface:

[A] Setup a directoryallowed for dumping rdf files
[A] Autogenerate documentation: trees, and labels. How do you test for correctness and completeness in a data store?
[C] guile bindings for lmdb for important stuff
[B] Using hashes to track updates on database---proposal

Alex:

Fetching all the phenotype data from the database using sql and genotype file
Using LMBD as input for correlations---make proposal for how we can export data in a way we can use

example implementation for this:

https://github.com/Alexanderlacuna/correlation_rust/pull/3/files#diff-87f0275ed65ae8ffe36df25091c9b975defbc920da7b5ec3c2dadb7d57ad4d67

Fred:

Fetch metadata in RDF and possibly document things