Text reflow woes (or: I want bullets back!)y

[solderpunk]

On Fri, Jan 17, 2020 at 08:52:52AM -0800, Aaron Janse wrote:
> 
> Definitely. Unless there is a problem I'm not seeing, this sounds very
> exciting!

I'm excited by this, too.  Because I can't control myself I've already
started writing up a rough spec for this, included below for comment.  I
definitely want to wait until more people have chimed in before getting
too excited, and I'd especially like to hear from all, or at least most,
of the authors of existing clients before making even a tentative
decision.  But I have to say, in writing up the below, I get a very good
feeling.  The description is not exactly short, but it's very *not
fiddly* compared to how it would look if we were going to support
"reflow in both directions".  This "half reflow" approach seems to hit
an amazing sweetspot of simplicity, richness and adaptability of display
widths.  I have figured out how to get vim to let me write this kind of
content in a pleasant and easy way, so I think I'm happy to switch.  And
if anybody *really* hates it, they can still hard-wrap their files and
wide clients following these rules will display it properly.  Narrow
clients will end up with a mess, but that's just the price people will
have to pay for sticking to their hard-wrapping guns.  Many of them will
probably be happy to dismiss mobile clients out of hand, anyway!

Feedback on the below very welcome!

Cheers,
Solderpunk

---------

The text/gemini syntax is inspired by and looks visually similar to
Markdown, but it is significantly simpler (and consequently less
powerful).  The syntax is strictly line-based: a file can be processed
one line at a time and each line can be handled in a way which does
not depend upon any previous or subsequent lines.  The only internal
state a rendering engine needs to maintain is a single boolean
variable (for preformatting mode, described below).

The lines of a text/gemini file come in 8 different types.  It is
possible to unambiguously determine the type of a line by only
considering its first three characters.  Knowing a line's type is the
only information necessary to know how to handle it correctly.

Just like Gemini's status code system is designed so that simple
clients can ignore the second digit of the code and still function
correctly, the text/gemini syntax is designed so that simple clients
can treat several different types of line identically and still
provide a usable representation of the document.  A bare-minimum
client need only recognise four different types of line.  These are:

1. PREFORMATTED TOGGLE LINES

Lines consisting of only and exactly three back ticks (```) are
preformatted toggle lines.  These lines should not be displayed to the
user and are instead used to toggle preformatted mode on and off (the
mode is off when the parser is intialised).  When preformatted mode is
on, the usual rules for identifying line types are suspended and all
lines should be unconditionally identified as PREFORMATTED TEXT LINES.

2. PREFORMATTED TEXT LINES

Preformatted text lines should be presented to the user in a monowidth
font exactly as they appear in the text/gemini file.  Lines longer
than the client's viewport must not be wrapped, leading or trailing
whitespace must not be removed, etc.  Handling of lines longer than the
viewport is client-specific.  Advanced clients may display a
horizontal scrollbar.  Simple clients may simply truncate the line.

3. LINK LINES

Lines beginning with the two characters => are link lines.  We all know
how these work by now.

4. TEXT LINES

All other lines are TEXT LINES.  They should be presented to the user
in a client-specific "pleasing manner".  Lines longer than the
client's viewport should be wrapped into multiple lines of a suitable
length.  Variable width fonts may be used.  Blank lines are a special
case of TEXT LINES and should be reproduced in the output.

It is important to realise that while Markdown, HTML, LaTeX and many
other document markup formats are "block based" or "paragraph based",
the text/gemini format is not.  Consecutive, non-blank lines of text
which are much shorter than the client's viewport should *not* be
combined into longer lines.  Equivalently, *every* newline character
in a text/gemini file is significant, not only consecutive pairs of
newline characters.  Clients ignoring this fact will produce incorrect
output.

Advanced clients may also recognise the following additional line
types.  Simple clients may treat all line types below as TEXT LINES
with no loss of essential function.

5. HEADING LINES

Lines beginning with one, two or three consecutive # characters are
HEADING LINES, corresponding to headings, subheadings or susubheadings
respectively.  The text following the # characters (with leading
whitespace removed) constitute the heading and should be displayed to
the user.  Clients MAY choose to apply special styling to headings to
distinguish them from ordinary lines.  However, the primary purpose of
HEADING LINES is to represent the internal structure of the document
in a machine-readable way.  Advanced clients can use this information
to, e.g. display a hierarchically formatted "table of contents" for a
long document in a side-pane, allowing users to easily jump to
specific sections without excessive scrolling.  Or CMS-style tools
automatically generating menus or Atom/RSS feeds for a directory of
text/gemini files can use the first heading in a file as a
human-friendly label for links to it.

6. UNORDERED LIST ITEMS

Lines beginning with a * are UNORDERED LIST ITEMS.  This line type
exists purely for stylistic reasons.  The * may be replaced in
advanced clients by a bullet symbol.  Any text after the * character
should be presented to the user as if it were a TEXT LINE, i.e.
wrapped to fit the viewport.  Advanced clients can take the space of
the bullet symbol into account when performing wrapping and ensure
that all lines of text corresponding to the item are aligned with one
another.

7. ORDERED LIST ITEMS

As above with obvious changes.