[SPEC] Encouraging HTTP Proxies to support Gemini hosts self-blacklisting

[Johann Galle]

Hi,

why is robots.txt not the obvious answer here? The companion specification[1] has a "User-agent: webproxy" for this specific case:

 > ### Web proxies
 > Gemini bots which fetch content in order to translate said content into HTML and publicly serve the result over HTTP(S) (in order to make Geminispace accessible from within a standard web browser) should respect robots.txt directives aimed at a User-agent of "webproxy".

So this should suffice:

```
User-agent: webproxy
Disallow: /
```

Regards,
Johann

-- 
You can verify the digital signature on this email with the public key
available through web key discovery. Try e.g. `gpg --locate-keys`...
or go to
<https://qwertqwefsday.eu/.well-known/openpgpkey/hu/spd3xecxhotzgyu1p3eqdqdp31ba6rif>.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xA4EFCC5A6174FB0F.asc
Type: application/pgp-keys
Size: 3131 bytes
Desc: not available
URL: <https://mailing-list-archive.gemi.dev/www/attachments/20210221/1221b622/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://mailing-list-archive.gemi.dev/www/attachments/20210221/1221b622/attachment-0001.sig>