An outsider's view of the `gemini://` protocol

[Ciprian Dorin Craciun]

On Fri, Feb 28, 2020 at 4:44 AM Sean Conner <sean at conman.org> wrote:
>   I disagree.  Using "proper symbols" is over all harder to deal with.
> First, it tends to be English-centric.  I mean, we could go with:
>
>         defectum:tempus:tardius
>
> or how about
>
>         teip:sealadach:níos-moille


The protocol is already English centric, for example the MIME types
(which are IANA standards), it uses lef-to-right writing, it uses
UTF-8 which is optimized for Latin-based alphabets, etc.;  so if we
want to be politically correct, we could use Latin or Esperanto.


>   First off, the code has to be parsed, and while this is easy in languages
> like Python or Perl, you run into ... issues, with Rust, C++ or Go (not to
> mention the complete mess that is C).  A number is easy to parse, easy to
> check and whose meaning can be translated into another language.  The Gemini
> status codes (as well as HTTP and other three-digit status codes) don't even
> have to be converted into a number---you can easily do a two level check:
>
>         if (status[0] == '2')
>                 /* happy path */
>         else if (status[0] == '3')
>                 /* redirection path */
>         else if (status[0] == '4')
>                 /* tempoary failure */
>         else if (status[0] == '5')
>                 /* permanent failure */
>         else if (status[0] == '6')
>         {
>                 /* authorizatio needed */
>                 if (status[1] == '1')
>                         /* client cert required */
>                 else if (status[1] == '3')
>                         /* rejected! */
>         }


OK, although I understand why things are harder in C, you present
above only the "easy part".  Please take into account the
line-reading, splitting into code and meta (and the protocol does say
one or multiple whitespaces in between), checking the `CRLF` at the
end.  Now assuming you've done all that even the code above has a
couple of bugs:
* what if the server sends `99`?  (it is not covered);
* what if the server sends just `6`? (it is not covered, although
given that perhaps `status` is `\0` terminated it won't be a large
problem, but still it would fall through;)
* what if the server just sends an empty status code? (is it checked
by the parser?)

So if simplicity is a real concern, then why not introduce something
like `0:success` `1:failure:temporary`.  (I.e. the first character is
either `0` or `1`;  other more advanced clients should parse the
rest.)

Also taking into account that the client still has to handle relative
redirects, I think the status code parsing pales in comparison.


As minor issues:
* why `CRLF`?  it's easier (both in terms of availability of functions
and efficiency) to split lines by a single character `\n` than by a
string;
* why allow "one-or-more whitespaces" especially in protocol related
parts?  why not mandate a strict syntax?




> > Now on the transport side, somewhat related to the previous point, I
> > think TLS transient certificates are an overkill...  If one wants to
> > implement "sessions", one could introduce
>
>   This is the fault of both myself and solderpunk.  When I implemented the
> first Gemin server (yes, even more solderpunk, who created the protocol) I
> included support for client certificates as a means of authentication of the
> client.  My intent (besides playing around with that technology) was to have
> fine grained control over server requests without the user to have a
> password, and to that end, I have two areas on my Gemini server that require
> client certificates:
>
> [...]
>
> It wasn't my intent to introduce a "cookie" like feature. solderpunk
> interpreted this as a "cookie" like feature and called it "transient
> certificates".  I still view this feature as "client certificates" myself.
> I personally think the user of "transient certificates" is confusing.


I was specifically targeting only the "transient certificates", not
proper "client certificates".

In fact I appreciate very much the usage of client certificates as
means to authenticate known clients.  (This is something I personally
use in production for back-office endpoints.)




> > On a second thought, why TLS?  Why not something based on NaCL /
> > `libsodium` constructs, or even the "Noise Protocol"
> > (http://www.noiseprotocol.org/)?
>
>         1) Never, *NEVER* implement crypto yourself.


I was never proposing to implement crypto ourselves.  `libsodium` /
NaCL provides very useful high-level constructs, tailored for specific
use-cases (like for example message encryption and signing), that are
proven to be safe, and exports them with a very simple API that can be
easily understood and used.


>         3) I never even heard of the Noise Protocol.


The "Noise Protocol" is currently used by WireGuard, WhatsApp and
possibly other applications that target network-based communications.
Although it is more complex than NaCL.

(It was just an example of more "current" frameworks.)


>         2) OpenSSL exists and has support in most (if not all) popular
>         languages.


Don't know what to say...  I find the OpenSSL documentation terrible,
and it's hard to use...  In fact given the complexity of TLS I would
say any wrapper, reimplementation, or alternative is as bad.  For
example I played with Go's TLS library and even though it's manageable
it requires lots of attention to get things right.




> > For example I've tried to build the
> > Asuka Rust-based client and it pulled ~104 dependencies and took a few
> > minutes to compile, this doesn't seem too lightweight...
>
>   So wait?  You try to use something other than OpenSSL and it had too many
> dependencies and took too long to compile?  Or is did you mean to say that
> the existing Rust-based client for OpenSSL had too many dependencies?  I
> think you mean the later, but it could be read as the former.


Looking in https://tildegit.org/julienxx/asuka/src/branch/master/Cargo.toml
apparently it is using `native-tls`
(https://crates.io/crates/native-tls) which apparently is using
OpenSSL on Linux; and this `native-tls` library isn't an "odd" one, it
is used by many high profile Rust libraries.  Removing it and checking
the dependency tree it seems it drops the dependencies with about 15
packages.

However as said earlier, perhaps it's Rust's ecosystem fault, most
likely other used libraries might also be to blame for this, but
regardless mandating the use of TLS doesn't simplify things.




> > Why not just re-use PGP to sign / encrypt requests and replies?  With
> > regard to PGP,
>
>   There are issues with using PGP:
>
>         https://latacora.micro.blog/2019/07/16/the-pgp-problem.html


There are issues with any technology, TLS included.

However I would say it's easier to integrate GnuPG (even through
subprocesses) in order to encrypt / decrypt payloads (especially given
how low in count they are for Gemini's ecosystem) than implementing
TLS.  Moreover it offers out-of-the-box the whole client side
certificate management, which adding to a TLS-based client would be
much more involved, more on this bellow...


> > given that Gopher communities tend to be quite small,
> > and composed of mostly "techie" people, this goes hand-in-hand with
> > the "web-of-trust" that is enabled by PGP and can provide something
> > that TLS can't at this moment: actual "attribution" of servers to
> > human beings and trust delegation;  for example for a server one could
> > generate a pair of keys and other people could sign those keys as a
> > way to denote their "trust" in that server (and thus the hosted
> > content).  Why not take this a step further and allow each document
> > served to be signed, thus extending this "attribution" not only to the
> > servers, but to the actual contents.  This way a server could provide
> > a mirror / cached version of a certain document, while still proving
> > it is the original one.
>
>   The hardest problem with crypto is key management.  If anything, key
> management with PGP seems more problematic than with OpenSSL and the CA
> infrastructure (as bad as the CA infrastructure is).


One of the `gemini://` specifications explicitly states that the
server certificate authentication model is similar to SSH's first use
accept and cache afterward.  However say you'll go with the actual CA
model, now you need to juggle Let's Encrypt (each 3 months) (or add
support for ACME in your server), then juggle PEM files, etc.
Regardless, either way one will have to implement all this certificate
management from scratch.

Now on the client certificate side, again a client would have to
implement all that from scratch.

Thus on the contrary, PGP (with perhaps GnuPG) would simplify all this
because it already implements all these features, and has clearly
defined operations over all these entities, including a web-of-trust.

(In fact none of the package managers I know of use S/MIME, i.e. X.509
certificates and CA's, for package signatures, but instead delegate to
GnuPG...)




> > Now getting back to the `gemini://` protocol, another odd thing I
> > found is the "query" feature.  Gemini explicitly supports only `GET`
> > requests, and the `text/gemini` format doesn't support forms, yet it
> > still tries to implement a "single input-box form"...  Granted it's a
> > nice hack, but it's not "elegant"...  (Again, like in the case of
> > sessions, it seems more as an afterthought, even though this is the
> > way Gopher does it...)
> >
> > Perhaps a simple "form" solution would be better?  Perhaps completely
> > eliminating for the time these "queries"?  Or perhaps introducing a
> > new form of URL's like for example:
> > `gemini-query:?url=gemini://server/path&prompt=Please+enter+something`
> > which can be served either in-line (as was possible in Gopher) and /
> > or served as a redirect (thus eliminating another status code family).
>
>   Forms lead to applications.  Applications lead to client side scripting.
> Client side scripting leads to the web ...
>
>   Of course there's pressure to expand the protocol.  solderpunk is trying
> his hardest to keep that from happening and turning Gemini into another web
> clone.


But you are already implementing "applications" on-top of Gemini (and
Gopher) through CGI...  And you already are implementing forms,
although "single-input" ones...  Even with this single input form one
could implement a wizard style "one input at a time" form...
Basically you give the technical possibility for "applications".

I wasn't talking about "client side scripting";  I was just saying
either drop this completely from the protocol, either specify it.  (At
the moment nothing stops a client / server implementer to just reuse
the "question" and "answer" to send back and forth an actual form
specification and answer...)

(Also "client side scripting" can't be eradicated through the
protocol.  One is free to include for example JavaScript in the
client, and the protocol can't say "no".)




> > Regarding the `text/gemini` format -- and taking into account various
> > emails in the archive about reflowing, etc -- makes me wonder if it is
> > actually needed.  Why can't CommonMark be adopted as the HTML
> > equivalent, and a more up-to-date Gopher map variant as an alternative
> > for menus?  There are already countless safe CommonMark parsers
> > out-there (for example in Rust there is one implemented by Google) and
> > the format is well understood and accepted by a large community
> > (especially the static side generators community).
>
>   It can.  RFC-7763 defines the media type text/markdown and RFC-7764 define
> known variations that can be specified.  Could be done right now without any
> changes to Gemini.  Go for it.


I know "I can";  I can even use PDF as the default "document format"
in my own client / server.  I could even use Flash.  :)

However I was speaking as the "default", Gemini endorsed format.




> > Regarding an up-to-date Gopher map alternative, I think this is an
> > important piece of the Gopher ecosystem that is missing from today's
> > world:  a machine-parsable standard format of indexing documents.  I
> > very fondly remember "directory" sites of yesteryear (like DMOZ or the
> > countless other clones) that strives to categorize the internet not by
> > "machine learning" but by human curation.
>
>   Could you provide an example of what you mean by this?  I'm not sure why a
> map alternative is needed.


One problem with today's web is that the actual "web structure" is
embedded in unstructured documents as links.  What I liked about
Gopher maps is that it gave a machine-readable, but still
user-friendly, way to map and categorize the "web contents".

Think about the following example:  I want to look for a cheap telecom
plan;  I open multiple telecom provider web sites, and now for each
one I have to "navigate" their "UX optimized" layouts (expanding
menus, drop-downs, burger buttons, etc.) (some placed on the top, some
on the right, etc.) to find the proper page that lists these plans.
Now imagine how that looks in Gopher:  each site would in fact provide
a Gopher-map that looks the same (at least in terms of layout) and I
can find the information I'm looking for much easier.

To be more "academic":  the current web pages (HTML) serve a couple of
distinct purposes:
* actual documents -- documentation, essays, blogs, etc. that use
links mainly as bibliographic references;  (this would be equivalent
to PDF files;)
* actual client-side applications -- single page apps (React, Angular,
etc.);  (this would be equivalent to Flash;)
* index pages -- meant to give a structure to a particular site and
help the user find what he is searching for;  (this would be
equivalent to site-maps, Gopher maps, RSS, etc.)

Now getting back to Gemini:
* actual documents -- we have `text/gemini` (or CommonMark, etc.)
* we don't support client-side applications;
* index pages -- we once more have `text/gemiin` (or others) but which
aren't specifically designed for this purpose;

How would such an "index" document look like?  A machine readable
(don't have the specific syntax yet, perhaps JSON?, perhaps something
else?) that allows one to:
* provide a way to describe links with short description (perhaps a
summary, author, date, some other standard meta-data like RSS/Atom
does);
* provide a way to group links in a hierarchical tree-like menu (so
that one doesn't need multiple transactions to load a small depth,
well structured menu);
* provide hints for crawlers, etc.;




> > * and perhaps add support for content-based addressing (as opposed to
> > server-based addressing) (i.e. persistent URL's);
>
>   There already exist such protocols---I'm not sure what a new one based
> around Gemini would buy.


I agree that `gemini://` is first and foremost a "transfer" protocol.
However one can include a document's identity as a first class citizen
of the protocol.

For example say each document is identified by its SHA;  then when
replying with a document also send that SHA in form of a permanent URL
like say `gemini-object:?sha={SHA}&location=gemini://first-server/...&location=gemini://second-server/...`;
 then a client (that perhaps has bookmarked that particular version of
that document) could send that URL to a server (of his choosing via
configuration, to the first one specified in `location`, etc.) and if
that server has that document just reply with that, else use
`location`, else return 404.

Ciprian.