Binding notebooks

Binder and Cocalc are tools for deploying Jupyter notebooks, R/shiny notebooks and Pluto notebooks. These binding tools do a lot more and do not leverage GNU Guix lightweight container infrastructure. In this project we want to create a binding-lite.

Tags

Members

Kanban

Implementation

Key is to leverage what is in Guix and what we already have in GeneNetwork as much as possible.

We should create this separately from genenetwork because all we want to do is run notebooks in containers. We should be able to use and external authentication system, fetch notebooks from git elsewhere and push them on git elsewhere.

binding-lite will be packaged in GNU Guix. GeneNetwork will essentially be a 'rebranding' of binding-lite.

Later we'll add an API to create notebooks on the fly. That is a separate topic.

Background

The current way of using guix-jupyter is to create a guix environment that contains the guix-jupyter kernel and jupyter.

The user can then run the `jupyter notebook` command which prints a url to the terminal for the user to copy/paste into the browser.

Once the url is opened in the browser, a user has to select the guix kernel and choose the notebook from the file manager menu.

Initial Design Ideas: guix-notebook-launcher

The first goal is to fire up a jupyter notebook from a local git repository.

Afterwards, this would be implemented with a remote git repository.

We need a command line interface/tool and guile library to automate the process of opening a jupyter notebook from a local git repository.

Some of the tasks that `guix-notebook-launcher` will need to automate is preselecting the guix kernel and opening the browser to the notebook view

of the given url.

Here is an example of how the command line interface would be used:

$ guix-notebook-launcher file:///path/to/<git-repository>

To access the notebook, open this file in a browser:
        file:///path/to/<git-repository>
    or copy and paste one of these URLs:
        http://localhost:8890/?token=fc21c17b9ec31eddbea28c06ce086cf71b5605a09610439b
    or  http://127.0.0.1:8890/?token=fc21c17b9ec31eddbea28c06ce086cf71b5605a09610439b

Managing Jupyter Notebook Dependencies

We'll need a manifest.scm file that will specify the dependencies needed by the guix-jupyter notebook.

Below is an example of the contents of a guix-jupyter notebook repository that can be given to `guix-notebook-launcher`:

$ tree <git-repository>
.
├── LICENSE
├── README.md
├── index.ipynb
└── manifest.scm

An example of the contents of manifest.scm:

(specifications->manifest
  '("python-numpy" 
    "python-pandas"
    "python-matplotlib"))

Additional Features and Ideas

One possibility for easing the creation of a manifest.scm in the notebook repository is to parse the requirements.txt/setup.py,

in the case of a python notebook, and generate the manifest.scm file for the notebook user.

We would have to write similar manifest generators for Julia and R project dependencies in order to support this feature.

I will ask on the guix mailing lists for feedback on this idea.

Once we have this library we can start building binder-lite on top of it.

Use Cases

The use cases for `guix-notebook-launcher` are driven by the following design decisions:

`guix-notebook-launcher`, by users on their workstations.

Initial Ideas for binder-lite Front End and Backend

Next Development Steps

Current Notebook Launcher Issues

store-connection-error

Current Notebook Container Default Settings

Current WIP Code

binderlite demo

binderlite example repository

Script / Command Line Interface / Web UI Mock / GNU Artanis

Reference Links

Jupyter Notebook Config File

Gesis Notebooks Front Page

GNU Artanis

Scheme Language in Production

Binder Example

Current Questions I Have to Answer

Yes

A manifest! Which of course is just another record :)

scheme@(guix-user)> ,use (gnu packages) (guix profiles)
scheme@(guix-user)> (manifest? (specifications->manifest '("python-ipython")))
$1 = #t

Reference Links

Guix Jupyter Magics

Misc Binder Lite Ideas

Should binder-lite be a distribution of GNU Guix System?

GeneNetwork can provide custom ISO and QEMU images that contain binder-lite ready to be deployed to bare metal or a VM.

This is inspired by the-littlest-jupyterhub distribution.

Reference Links

The Littlest JupyterHub

Basic Working Demo

Demo

TODOs