2021-08-07

Re: Think in Writing

skyjake writes:

Programming is such a complex task that it helps immensely to write things down. At the end of a session one can just dump one's thoughts and next steps into a text file, and pick them up the next day, or after a month, getting back up to speed quickly.

gemini://skyjake.fi/gemlog/2021-08_think-in-writing.gmi

local copy

I can second, and I cannot recommend it enough. So I'll write down, how this works for me. It is not limited to programming.

---

Somewhere I read, that apparently Richard Feynman insisted, that the thinking happens on paper [a] --- what is not written down, does not really exist. And as a physicist by education myself, I can support that. If you cannot write it down somehow, it might be lost.

---

I keep a detailed log of my work and my thinking at $dayjob. I use emacs/org-mode for that. I keep accounting of time to specific projects in org-mode, albeit in a separate file. I also keep private and work files completely separate.

The Journal

So, I have ONE file called "journal.org". It is a file+datetree+prompt type of org journal. I started this file on my first day of this $dayjob five years ago, and I keep adding to it. The file holds more than 200000 lines (yes, including all the empty ones!). I have no intention to start a new file, or one file per year or something such. Allthough company-mode starts to become slow, and starting consult-line takes more than two seconds to start, I cannot see any advantage in using smaller files.

Structure

I have written a capture template to start any new day. Actually there are five separate templates, one for each weekday. They hold slightly different entries, e.g. a checkbox to update my accumulated times on Monday morning, and checkboxes for my two regular meetings on Thursday.

There are more capture templates and yas snippets to document recurring tasks, mostly consisting of lists of org-mode headers and checkbox lists.

Content

What goes into the file? Lots of things.

A brief entry about whether anything special happened during commute. Mostly one line.

A header type section about todays/every days things to check.

Notes about administrative things, like opening a support ticket or dealing with infrastructure.

Every project gets a separate header with project numbers as properties. This I do with yasnippet, but could be done with capture templates equally well.

On a typical day I will start with such a project header and often I'll add "Where was I?". I'll collect the status of this projects puzzle pieces into a description --- even if I left it yesterday evening. Writing this down sets the context I need to activate in my brain. Imho this becomes more important with age.

Whatever I decide to do next I write down. Then I start. I will add pieces of information like which branch in which git repository, or which login on which system in which directory this happens. And this has saved me from hours of searching and head scratching.

Most importantly I will add org-babel sections with shell commands, and example sections with output of programs I use. This serves me well: I can often find the exact steps how I did something last time. I can often find the error message currently staring at me on the other terminal shell plus some explanation on how to proceed. I can find out that it is in fact four years ago that I did some task for the first time.

When I'm out of ideas, I will write this down. I will add what I tried, and why it failed. And the very act of writing this stuff down --- similarly to what you would need to spell out to your support person --- will often trigger new or lost insights, or will raise sudden doubts about what you believed to be certain two minutes ago. I would agree, that this process is close to "rubber ducking", just not by voice, but by typing. It has happened to me that I stumble over the source of a problem by writing an email call for help. Also see below for another view on this process [b].

Once a week I collect together a status report. In my role I have several project managers I need to report to, not only my manager and his manager. I make this report quite detailed including links to tickets and documentation, such that the recipients can navigate to the context I mention with ease. I do not expect every recipient to read all of the report. But this report is an important source of information. So I write it up in my logfile, and copy/paste it into the status email.

Other

Some things go into separate .org files. Mostly some kind of detailed reports that I write to document the status quo, Howtos of tedious or complex steps and the like. Some of this goes into $document-system, $email, $source-code-repository etc.

For a particular project I have tried org-roam (v1) and org-roam-server. While this tool is pretty phantastic, I have not made the switch to keep everything in org-roam, allthough version 2 now offers more than one "Zettel" in a file --- all headers with org-ids can be referenced and thus are now considered "Zettel".

I'm using org-mode TODOs, todo-captures, agenda view and the like to some extent. But I am not good at this. My preferred method of navigating the infinite pile of things to be done is this: in the morning, maybe on the train, I'll let my mind wander through the open things (as far as I remember them :-) and decide, which one looks like the most important (or sexy) thing for today. This decision might be moot the minute I walk into the office or open my mail box --- never mind. Needless to say, that this is not anywhere near "well organized". Sigh.

I have not tried to convice emacs to work with the ticket system. The number of tickets I'm involved with, is low --- and I hate tickets anyway.

I do use file links and occasionally org-attach.

I keep all of these files in ONE directory

I keep said direcotry in a git repository --- just in bloody case!

I totally fail to use TAGs, sparse views, and properties to a more useful extent. So org-mode is a lot bigger than I can handle it.

I am totally sold on the idea, that plain text is the most portable format. Tools like grep, ag, rg are a tremendous help. emacs/consult-line or swiper is my preferred tool to find things, rather than infile-linking.

Some numbers

In my current job (5.5 years up to now), I use org-mode pretty much exclusively. Of course I have to transfer information across to corporate systems about once a week.

2_journal.org 224645 lines, close to 9 MiB (as of 2021-08-06). approx. 3400 lines per month.

some 60 additional .org files: 16000 lines

In the job before (9 years total), I used org-mode for tracking time (not from the first day, though). But I had a journal as well. It was in ChangeLog format, and yes it is one file for the whole gig!

WorkLog: 119882 lines, approx. 4 MiB, approx 1100 lines per month.

Some 250 other files: 70000 lines

In the job before that (3 years total), I used emacs record.el to track times, and a ChangeLog type log file. But I don't have them any more.

tl;dr

Do not only keep notes, but context and details, too.
Think by typing!
Plain text is your friend!
grep and $editors equivalent is your friend!
VersionControlSystem and Backup are your friends!

You write these notes for your future self --- noone else!
Any writing to be read/used by others must go into different form.

Well, this turned out much longer than I expected --- thanks for your time!

Cheers,

~ew

Footnotes

[a] Probably in "How to take smart notes" by Söhnke Ahrens.

[b] Robert Pirsig -- Zen and the Art of Motorcycle Maintenance --- This book has a very interesting section about what happens, when you run out of ideas. And I totally agree with that description. Something like: you need to sit in front of the problem (broken motorcycle) and stare at it for a while. After some time a little fact will surface in your mind, knock on the door of conciousness and ask if you are interested. If you dismiss this fact, it will disappear and not surface again for a very long time. Been there, done that. However, if you listen to this thing, often it will lead you closer to the source of the problem. It's often some tiny fact that you thought is understood or done, when in fact it is not. And you have only this one chance.