💾 Archived View for gemini.circumlunar.space › users › emptyhallway › augr › text-reflow.gmi captured on 2021-12-05 at 23:47:19. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

(This is part of a "what if" series of idle thoughts. If you haven't read the introduction, please do that first.)

Text Reflow

Background

When an arbitrary program displays text within a viewport, when should it end the current line of text and start at the beginning of a new line? There are a few options:

When the text is longer than the width of the viewport, and would overflow the viewport area.
When the program encounters a new line character (which I won't define here, because that's its own familiar mess) in the text.
When the program encounters some other tag or marker in the text. For example, two consecutive new-line characters, or <br> as in HTML.

More specifically, what should Gemini clients do? This was a topic of discussion on the Gemini mailing list for several months in 2019–2020.

There are three options that seemed most popular, which I'll attempt to summarize:

Fixed viewport: The client must display 80 characters and no more. Authors must manually enter line breaks at appropriate points in their text to stay within the 80-character limit, or risk unspecified behavior.
Reflow text: Combine adjacent lines of text into paragraphs. (Ignore line breaks between adjacent lines of text.) The client must insert its own line breaks at appropriate places to fit the text inside the viewport. Authors indicate paragraph breaks with two consecutive new-line characters.
Wrap text: Start a new line when encountering new line characters in the text. The client must insert its own additional line breaks to avoid overflowing the viewport. Authors indicate paragraphs however they want: an empty line between text lines or (maybe?) by indenting the start of each paragraph.

In the end, the "wrap text" option won out. For the Gemini markup format, this has some advantages:

It's easier to author Gemini markup when line breaks are "natural". Manually splitting up paragraphs into lines of a fixed width is tedious work.
There is no single fixed width that works for everyone. Different platforms (terminals, GUI desktop clients, mobile clients) all might have slightly (or dramatically) different optimal viewport widths.
Not all text is paragraphs. For stylistic reasons (bullet lists, emphasis, poetry, code snippets, ASCII art), authors may want to break lines without starting a new paragraph. This requires a new syntax element in a "reflow text" environment, which increases complexity (both for parser programs and human authors).
Reflowing is complicated. Are the lines joined with a space between them or not?

The elegance of Gemini markup

One of the properties that I like a lot about Gemini markup is that it's entirely line-based. Each line is only one thing: a header, a link, normal text, or whatever. This makes it easy to read and easy to parse.

Someone on the mailing list (sorry, I haven't looked up who) made analogy to tag-based markup of HTML, except that the closing tag is always the same: a new line character.

But… reflow means inline markup!

Despite the simplicity of wrapped text, while I was reading through the mailing list discussion on this topic, I was slightly rooting for the reflow option to win out. It turns links into inline links! Under current Gemini markup, each link appears on a separate, dedicated line from adjacent text. This makes it impossible (or at least unaesthetic) to have an inline link, where a few words in the middle of a sentence are a link.

But in a reflow world, adjacent lines are joined automatically. A single-word link on a lne by itself is joined with adjacent lines before and after it. The result is an inline link.

Taking this a step further, text reflow also unlocks other inline styling. Current Gemini doesn't offer support for bold or italics. It probably wouldn't be very useful in a wrapped-text world, where an entire line has to be either plain, bold or italic. This isn't how bold or italic are used in typical writing.

But in a reflow world, bolded or italicized "lines" are merged, too, and simply become part of a larger paragraph. Suddenly, we can generate output that looks a lot more like HTML or Markdown, with inline links and emphasis.

For example, bold lines could begin with "_ " (underscore followed by space) and italic lines could begin with "/ " (forward-slash followed by space). Like all Gemini markup, they end with a new-line character.

What about non-paragraph line breaks?

Of course, if we are ignoring single new-line characters, what happens to authors who want to specify new lines without starting a new paragraph? Here are some of the options I saw discussed or considered myself. (In this section, I assume that "reflow" is the default and special markup is needed to disable it, but the inverse is equally valid.)

Introduce a trailing markup at the end of the line that indicates a line shouldn't reflow, like a backslash (as in many coding languages) or a space, which is less visually distracting.

This breaks the elegance of the Gemini markup, where you learn everything you need to know about a line by looking at the first three (or four) characters. Now you have to look at (or keep track of) the last character of the previous line to know how to handle each new line.

Introduce a new leading markup at the start of the line that indicates a line shouldn't reflow, like a single space, or something more visibly obvious.

This kind of breaks the whole reason I was excited about reflow. If you need markup to indicate whether to reflow text, then you can't use markup to indicate anything else (links, bold, italic). A more complicated markup scheme could allow this to combine with other line types, but this takes us farther from the simple elegance of Gemini markup again.

A new full-line markup that switches reflow mode on or off for following lines.

Like "```" (triple-backtick) toggles preformatted text mode, a new syntax could toggle between reflow mode and wrap mode. For example, ",,," (triple-comma) migh serve this purpose.

Of course, I would also combine this with my separate proposal to turn "line mode toggles" into "line mode starters". In that scenario, markup indicates the type of the upcoming lines, instead of turning a mode on or off. This allows an arbitrary number of different line modes, instead of only two.

Line mode markers

This fill-line markup approach combined with line mode starters is my favorite solution.

It's still too hard

Sadly, text reflow is still complicated. How do you handle bullets while in reflow mode? Do they have to be ignored? Do they automatically break reflow? Do you join lines with a space between them? Or not? Should it be predictable and simple (probably no automatic space, which means authors need to include their own trailing spaces) or smart (automatic spaces, except around punctuation or in languages that don't need it, which might not always be what the author wants).

All of these questions have solutions, but whatever answer you pick, it will increase the complexity of the parser, the cognitive burden of the author, and reduce the readability of the source.

So, I admit that wrapping text is the best compromise between flexibility and simplicity. It was the right choice for Gemini.

emptyhallway

2020-09-14