Color and escape codes in text/gemini

[Sean Conner]

It was thus said that the Great Baschdel once stated:
> In my last post on this list I called escape codes spaghetti without much
> of an explaination.
> 
> Tl;dr:
> -Do we want color in text/gemini?
> -If yes how should it be standardized?

  Oh goody!  We still haven't figured out our last question about Gemini:
To reflow or not to reflow, that is the question.  

> I really like that there are creative people out there looking for ways to
> push systems like gemini outside of what they are supposed to be able to
> do.

> And I don't want to decide (at least not now) if the escape code thingy is
> a "bug" or a feataure in most clients. The first question we have ask here
> is if we want color support at all: I know, that gemini should remain
> simple and support for colors takes away a good part of this simplicity.
> But there are a lot of usecases for color support. (Code highlighting,
> pretty ascii art, footnotes, ...) For me these are two pretty strong
> arguments against each other.
> 
> So in case we want color support: Assuming the escape code thingy is a
> feataure: We should standardize which escape codes are allowed and how
> clients should handle them. Assuming it is a "bug": Some people (me
> included after thinking about it) like to see color being used in gemini
> so we have to find another way to put colors in while the text/gemini
> format should remain easy to parse and human readable. (Correct me if I'm
> wrong but that is how I interpret the text/gemini format: Markup that is
> easy to parse but still human readable if you open it in something like a
> text editor)

  Ah, the so called ANSI escape codes, which aren't at all defined by ANSI
but ISO, and are technically known as ECMA-48.  First off, the control codes
that *are* defined by ASCII, codes 0 to 31 [1].  There are only seven
control codes that releate to text (we're excluding the unit separators,
which are rarely, if ever, used):

	07	BEL	audible alarm
	08	BS	move left one space
	09	HT	horizontal tab
	0A	LF	move down one line
	0B	VT	vertical tab
	0C	FF	form feed
	0D	CR	carriage return

  Of these, HT, LF and CR are the most used---the rest not so much in text
files.  Now technically, what we consider the backspace is actually the
action of two control characters, BS (move left one space) and DEL (ignore
this character---if things worked this way, an easy way to get, say, an
umlaut is 'a BS "' or an n with a tilde over it by 'n BS ~'), and there are
fights over how to end a line (CR, LF, or both?  Technically, Microsoft got
this right with both).

  Then there's ECMA-48, which is what is under discussion here.  It's a vast
standard, but they all fall under a few patterns (using a slightly modified
RFC-5234 format---"a" - "z" means the range of characters between "a" and
"z"):

	Pattern group 1 (largely, a few exceptions in here as shown below):

		group1 = %d27 ( "`" - "~")

	Pattern group 2 (largely, a few exceptions in here shown below):

		group2 = %d27 ( "@" - "_")
	               / %d128-175 ; [3]

	Pattern group 3 (these are what is popularly known as ANSI codes,
	and a subset of group 2):

		group3 = (%d27 "[" / %d155) *("0" - "?") *1(" " - "/") ("@" - "~")

	and finally pattern group 4 (and are a subset of group 1):

		cmd     = %d144 / %d152 / %d157 / %d158 / %d159
		        / %d27 ( "P" / "X" / "]" / "^" / "_")
		group4  = cmd *(%d8-13 / " " - "~") (%d156 / %27 "\")

  Group 4 is the most problematic, security wise, as they define actual
messages.  'ESC]' is defined as "OPERATING SYSTEM COMMAND" and 'ESC_' is
"APPLICATION PROGRAM COMMAND".  Not many terminal emulators support group 4,
but they are defined.

  And there are several commands under group 3 that are problematic.
Microsoft defined 'ESC[...p' (which is in the private use area of
ECMA-48) to redefine the keys on the keyboard (so beware of the text file
that does 'ESC[13;"deltree c:\"p').

And there are code that do more than just define colors, they can define
fonts, move the cursor, define locked regions on the screen, set or clear
tab stops, insert lines or characters on the screen, delete lines or
characters on the screen, and even define print parameters.  But oddly, no
defined sequence to query the size of the screen (fancy that!).

  Now, not all terminial emulators support all of ECMA-48, but it's a large
standard (and there is plenty of room for terminal emulators to extend
support with private codes).  

> If I had to standardize it I'd use custom escape sequences, similar to
> those already in use but optimized to be no longer than necessary that
> ALWAYS end in a ';'. 

  Hard to enforce, and the ending in ';' is problematic with respect to the
current standard.  For instance, to set the foreground color blinking and
the background color:

	ESC[31;42;5m

> This is not really text editor friendly, but avoids
> having to escape the escape caracter (I'm looking at you XML), makes it
> easy to parse/strip the escape sequences and if you don't you at least get
> very short spaghetti.

  So how would you define the control sequences?

  One thing I noticed with gemini://konpeito.media/ is that the default page
was very hard for me to read, but that's because the page made one
assumption that isn't universally true---that the default background color
is black.  I don't use a black background on my terminals.

> In any case I'm for filtering out unwanted escape codes.

  That's what I do, only I filter out all escape codes, since I don't want
the screen to mess up and my program get confused as to what is where.

  -spc

[1]	32, or space, is special in that it can be treated as a control code
	(part of the unit separator group starting with 28), or the
	graphical portion (even though it has no graphical representation). 
	And 127 is technically not part of any control group, as it
	technically means "ignore me entirely" [2].

[2]	It comes from paper tape, which originally were 7 bits long, and the
	hole represents a 1.  If there was an error, to fix it, all 7 bits
	were punched out (representing 127) and and reading side was known
	to ignore that character entirely.

[3]	If you are using the UTF-8 encoding scheme, these characters will be
	encoded as UTF-8 codepoints, so 155 is encoded as the byte sequence
	194,155, as if things weren't bad enough.