GEMMA permutations

by Pjotr Prins - 2021-07-16

Introduction

The existing version of gemma-wrapper handles permutations. It complicates the code and I think it better to have a wrapper around the wrapper. I reverted on the original idea, however, of naming it gema-wrapper-wrapper ;). The idea of running permutation and forking them out to slurm is pretty generic. The reason we have to write something new is that I want to feed progress back to GeneNetwork - significance is already visible at 100 permutations, so we should be able to show that as we go along. Computing the result of permutations ought to be a 'lambda' so the permutations tool is not specific to GEMMA. In fact, it may be a sheepdog tool. It needs some thought.

In this work I plan the following steps:

• [ ] Sanitize deploy (sheepdog), gemma-wrapper, gemma, gemma2lib projects by concerns
• [ ] Start a new tool for running permutations (sheepdog-permutations)
• [ ] Invoke locally
• [ ] Feed back progress on significance (through redis)

Gemma-wrapper

Sheepdog (original)

Gemma2lib

The slurm part we'll implement in another blog

Split tools into 'concerns'

Currently tools are doing too much or too little!

Start permutations

Invoke locally

Feedback progress through redis

Invoke slurm (other blog)