2023-09-20 Generating forward indexes for Oddµ

I find myself working on forward indexes and feeds for Oddµ. Or perhaps I should say: outside of Oddµ. I'm still holding off because it seems to be a very particular way of thinking about a wiki. My problem is that I use the wiki like a blog or Zettelkasten and therefore a feed is appropriate for my wiki.

Oddµ

So the first question is: Was I wrong when I wondered about feeds?

A feed is a dubious addition to a digital garden or a Zettelkasten.
Yes, the ideas can be put in a chronological order, but “following
along” is perhaps not the most efficient way of following along. –
To have a feed or not

To have a feed or not

If the wiki is a site where visitors come to see what’s up, then the site is a publishing platform more in line with news. This points at the wiki duality: a digital garden where people work on pages like they would on papers to be published later or not at all, linked to from elsewhere; and a blog with updates of thoughts and projects. I find myself using the wiki like a blog with updates, not like a digital garden.

In order to use the wiki like a blog, the only thing that really seems to be missing is forward indexes. A forward index in this context is simply a list of links. On the front page, a list of links to “new” pages. On a topic page, a list of links to “new” pages for that topic. In the feed, a list of links (and the complete page content, because that's how people read the feed).

Casual visitors to the site don’t know what the site has to offer.
When visitors arrive on the front page, they need to know what the
site is about, and they need to see a few interesting links to
follow. When visitors arrive on any other page, they want to explore
by following links.
– Forward Index

Forward Index

For Oddmuse, I used “journal pages”. These pages transcluded the last ten appropriate pages, resulting in that wall of text vibe that I like. This is also what many of the traditional blogs look like. However: If you want to find the older pages, you need to click on "Next" or "More" links, or you have to navigate an archive organised like a calendar. Neither is something I particularly enjoy.

Oddmuse

wall of text

What I did do a lot on my old wiki, however, was search for page titles and tags. Now, hopefully the search Oddµ has is good enough to offer that – we’ll see – but topic pages like RPG having a long list of links for easy browsing by page name seems like a good idea to me.

RPG

All of this to say that right now I have a Perl script that I call on the command-line. This is why I said I had a solution “outside of Oddµ”, at the top. I might call it as a CGI script one day, who knows.

a Perl script

For every file I give it as an argument, it does the following:

front page

For every hash tag the file contains, it does the following:

To populate new topic pages with links to existing pages, I have used the following pipeline from hell: `grep --ignore-case --files-with-matches '#Oddµ' wiki/*.md|sort --reverse|perl -e 'use Encode; use Mojo::Util qw(url_escape); use File::Slurper qw(read_text); binmode(STDOUT,":utf8"); while(<>) { chomp; $d=read_text($_); ($t)=$d=~/^# (.*)/m; s/\.md//; s/^wiki\///; $s=$_; $s=~s/_/ /g; $n=url_escape($_); print "* [$t]($n)\n"}'` … what can I say? I think it works. 🤪

For the front page, I only wanted to link the pages from 2023 as long as they had a tag so I started the pipeline with a different selection: `grep --ignore-case --files-with-matches '#' wiki/2023*.md`

​#Oddµ ​#Perl

Now I just have to add a way to semi-automate the other part: on a page, "publish" means to add a link to this page to all the existing "topic pages", i.e. on a page, look at all the hashtags, check if a page with that name exists, and if so, check if that page has a bullet list. If yes, prepend a link to the current page; if not, append a link to the current page. It might work. And when it does, the Perl script is no longer necessary.