Add gopher-extension.md. - gopher-protocol - Gopher Protocol Extension Project HTML git clone git://bitreich.org/gopher-protocol git://enlrupgkhuxnvlhsf6lc3fziv5h2hhfrinws65d7roiv6bfj7d652fid.onion/gopher-protocol DIR Log DIR Files DIR Refs DIR Tags DIR README DIR LICENSE --- DIR commit 7c5d9f434a37a64f1fd5b64991df8e154f1dd84d DIR parent f7bb7959972ce0d459cb2a7b478250f908b3728d HTML Author: Christoph Lohmann <20h@r-36.net> Date: Mon, 8 Aug 2022 18:27:05 +0200 Add gopher-extension.md. Thanks Evil_Bob for noting. Diffstat: A gopher-extension.md | 364 +++++++++++++++++++++++++++++++ 1 file changed, 364 insertions(+), 0 deletions(-) --- DIR diff --git a/gopher-extension.md b/gopher-extension.md @@ -0,0 +1,364 @@ +Gopher Extension +================= + +# Goals of this document + +The intention is to not make radical changes to the RFC1336 standard. + +This document also describes the common-used extensions to the +Gopher RFC and some clarifications to the wording of the RFC. + +Since the publication of the RFC1436 standard around March 1993 there +have been developments, such as the adoption of the UTF-8 +text-encoding and the use of SSL and later TLS encryption. + +The recommendations can be therefore be seen as guidelines or +"SHOULD". + + +# Added Types + +Types can be added, this doesn't violate the RFC specification: +section 3.8: "Characters '0' through 'Z' are reserved.". + +These are types that are commonly used. + +* The 'h' type: HTML file, with the "URL:" prefix in the selector it points to + an URL, see historical mail conversation (embedded below). +* The 'i' type, Informational message: display as text. + i Some message <TAB> empty selector server TAB port CR LF + The server and port should be included for compatibility. +* As mentioned in the original Gopher RFC, for other types: + Anything primary Text file? Use the 0 type. + Anything unknown or binary file? Use the 9 type and a file extension. +* Use the image (I) type for png, jpg etc. Make sure to set the file extension. + + +# Using the proper type for Text file or binary + +* Sometimes a question comes up where PDF or XML is binary. If the + file is readable as text it is a text file, otherwise it is binary. + + For example PDF would be using the binary 9 type and a .pdf file + extension. XML would be a text 0 type. + + Type 0 is files which are pure text and can be displayed in a text + editor. + + +# Text Encoding + +* The Notes section in the Gopher RFC mentions Latin1 encoding. + + Recommendation: Use UTF-8 or ASCII-only for the Gopher + username/title field. A client may want to display the other fields, + so be polite and use UTF-8 or ASCII there as well if possible. + + Reason: UTF-8 is a simple text-encoding and commonly used these days. + + People who use Latin1 eat children. + + +# Accessibility + +* From the Gopher RFC standard: + "It is *highly* recommended that the User_Name field contain only + printable characters, since many different clients will be using + it. However if eight bit characters are used, the characters + should conform with the ISO Latin1 Character Set. The length of + the user-displayable line should be less than 70 characters; longer + lines may not fit across some screens." + + New Recommendation: Don't use longer than 79 columns of UTF-8 encoded displayed + "username" text. + New Recommendation: Try to reduce the amount of ASCII art which can contain + non-printable characters. Think of the blind or tools used to parse actual + textual content. + Reason: A clarification of the term characters is needed. + +* "The selector string should be no longer than 255 characters." + + Recommendation: use no longer than 255 bytes. + Reasons: + * A clarification of the term characters is needed. "characters" could be + interpreted as columns. + * Clients can simply use a static buffer to fit 255 bytes. + * Although Gopher does not have to map to a filesystem, filesystems typically + have a limit of around 255 bytes also. + +* From section 3.5: + "If a client does not understand what a, say, type 'B' item (not a core + item) is, then it may simply ignore the item in the directory + listing; the user never even has to see it. Alternatively, the item + could be displayed as an unknown type." + + Recommendation: For clients, do not silently ignore an item, but display it + as an unknown type. + Reason: Define a recommendation for consistent behaviour in clients. + + +# Server and client handling of text file types + +The RFC defines: + + "Textfile Entity + + TextFile ::= {TextBlock} Lastline" + + "Note: Lines beginning with periods must be prepended with an extra + period to ensure that the transmission is not terminated early. + The client should strip extra periods at the beginning of the line." + + "Note: The client should be prepared for the server closing the + connection without sending the Lastline. This allows the + client to use fingerd servers." + + From section 4: + + "(b) The well-tempered server ought to send "text" (unless a file + must be transferred as raw binary). Should this text include + tabs, formfeeds, frufru? Probably not, but rude servers will + probably send them anyway. Publishers of documents should be + given simple tools (filters) that will alert them if there are any + funny characters in the documents they wish to publish, and give + them the opportunity to strip the questionable characters out; the + publisher may well refuse. + + (c) The well-tempered client should do something reasonable with + funny characters received in text; filter them out, leave them in, + whatever." + +The above description we think is too vague and it can be simpler. + +Recommendation: handle retrieving text file types the same as binary types. +For clients the Lastline pattern (".\r\n") is not handled specially in this case, +it is part of the data. +For servers no preprocessing is done on the TextFile data. + +Reason: Simplify the implementation of handling text types. Make the behaviour +of text output consistent for clients. + + +# The 'h' type: h_type.txt + +Below is an archived conversation about the Gopher 'h' type: + + "Received: with LISTAR (v1.0.0; list gopher); Tue, 12 Feb 2002 14:19:47 -0500 (EST) + Return-Path: <jgoerzen@complete.org> + Delivered-To: gopher@complete.org + To: gopher@complete.org + Subject: [gopher] Links to URL + From: John Goerzen <jgoerzen@complete.org> + Date: 12 Feb 2002 14:19:46 -0500 + Content-type: text/plain; charset=us-ascii + Content-Transfer-Encoding: 8bit + + I think it is best to start small with modifications to the protocol. + Therefore, I propose the following: + + Method to link to URLs from Gopherspace + --------------------------------------- + + 1. Protocol issues + + Links to URLs from a gopher directory shall be defined as follows: + + Type -- the appropriate character corresponding to the type of the + document on the remote end; h if HTML. + + Path -- the full URL, preceeded by "URL:". For instance: + URL:http://www.complete.org/ + + Host, Port -- pointing back to the gopher server that provided + the directory for compatibility reasons. + + Name -- as usual for a Gopher directory entry. + + 2. Conforming client requirements + + A client adhering to this specification will, when it sees a Gopher + selector with a path starting with URL:, interpret the path as a URL. + It will ignore the host and port components of the Gopher selector, + using those components from the URL instead (if applicable). + + 3. Conforming server requirements + + A server with Gopher URL support will not, in most cases, need to take + extra steps to provide this support beyond those outlined in + Compatibility below. Servers not implementing those steps outlined in + Compatibility will be deemed to be not in compliance. + + 4. Authoring compliance + + The use of URL: selectors should be avoided wherever possible. In + particular, it should be avoided when pre-existing gopher facilities + exist for the type of content linked. The following URL types are + explicitly prohibited by this specification: + + gopher + telnet + tn3270 + + Authors should avoid links to any document not of HTML type whenever + possible. Linking to non-HTML documents will break compatibility with + Gopher browsers that do not implement this specification. The ranks + of these browsers include most Web browsers, so that is a significant + audience. + + 5. Compatibility + + Links to HTML pages may be accomodated even for non-comforming + browsers by providing additional capabilities in the server. + + When a non-conforming browser is instructed to follow a link to a URL, + it will contact the Gopher server that provided the menu (since these + are specified per section 1). + + When a conforming Gopher server receives a request whose path begins + with URL:, it will write out a HTML document that will send the + non-compliant browser to the appropriate place. One such conforming + document is: + + <HTML> + <HEAD> + <META HTTP-EQUIV="refresh" content="2;URL=http://www.acm.org/classics/"> + </HEAD> + <BODY> + You are following a link from gopher to a web site. You will be + automatically taken to the web site shortly. If you do not get sent + there, please click + <A HREF="http://www.acm.org/classics/">here</A> to go to the web site. + <P> + The URL linked is: + <P> + <A HREF="http://www.acm.org/classics/">http://www.acm.org/classics/</A> + <P> + Thanks for using gopher! + </BODY> + </HTML> + + This document may be any desired by the server authors, but must + adhere to these requirements: + * It must provide a refresh of a duration of 10 seconds or less + * It must not use IMG tags, frames, or have any reference whatsoever + to content outside that particular file -- other than the link + to the real destination. + * It must not use JavaScript. + * It must adhere to the W3C HTML 3.2 standard. + + When a non-conforming Gopher client finds a reference to a HTML file + (type h), it will open up the file via Gopher (getting the redirect + document) but using a web browser. The web browser will then be + redirected to the actual link destination. Conforming clients will + follow the link directly. + + END + + + Comments? + +/h_type + + +# TLS support + +From: 2020-06-07 Gopher TLS prototype in geomyidae by 20h at +gophers://bitreich.org/0/usr/20h/phlog/2020-06-07T18-28-23-863932.md + + # 2020-06-07 18:28:23.863932 UTC (+0000) + + Gopher TLS prototype in geomyidae + + We are happy and proud to announce, that there is now a prototype of + gopher tls in geomyidae + + git://bitreich.org/geomyidae + + How does it work? + + When a client tries to connect via TLS, the first byte of the packet + will be 0x16 or 22 decimal, which is forbidden as a selector in Gopher. + This gives the server a hint to start TLS. Old servers will simply + reject such a connection attempt. + + For now clic supports TLS. We are working on hurl TLS support. And for + sacc it is on its way. + + git://bitreich.org/clic + git://bitreich.org/sacc + git://codemadness.org/hurl + + Hopefully further support will come to other clients. + + If you do not have anything at hand, here are some commandline clients: + + Plain old Gopher: + + printf "/\r\n" | nc bitreich.org 70 + + And with TLS: + + printf "/\r\n" | socat openssl-connect:bitreich.org:70,verify=0 - + + Have fun using TLS on gopher! + + + All patches and recommendations are welcome. + + + Sincerely yours, + + 20h + Senior Security Manager (SSM) + +/post by 20h + +# Gopher TLS URI + +A gopher TLS URI is the same as the Gopher URI described in RFC4266, +except the protocol scheme is gophers://. + +When the client using the Gopher protocol does not support TLS it can +simply use a plain gopher:// connection. + + +# Gopher TLS downgrades + +A client COULD implement the following logic: + +When a user uses gophers:// then it should use TLS and not downgrade +automatically to a plain connection. The client COULD also show a +_clear_ message if the TLS connection is not accepted and offer a +manual downgrade option to plain-text. + +When further selectors of the same host and port are accessed it should use +TLS automatically as well. + + +# Gopher+ compatibility + +Gopher+ allows adding more TAB-separated fields to the output. For +Gopher, to be compatible with Gopher+ clients, it can simply accept the +line, but ignore these additional fields. + + +# Other references: + +* RFC1436 - The Internet Gopher Protocol + https://www.rfc-editor.org/rfc/rfc1436.txt + or references/rfc1436.txt + +* RFC4266 - The gopher URI Scheme + https://www.rfc-editor.org/rfc/rfc4266.txt + or references/rfc4266.txt + +* Gopher+: + https://github.com/gopher-protocol/gopher-plus/blob/main/gopherplus.txt + or references/gopherplus.txt + +* geomyidae Gopher server: + git://bitreich.org/geomyidae + +* Helper tool to validate gopher and DirEntities: + git://bitreich.org/gopher-validator +