TEXT View source
# 2024-12-17 - No DNS, No Problem
The early Internet used /etc/hosts files to resolve names. These
were distributed on FTP sites. You can see a 1992 example below.
TEXT host.txt from textfiles.com
The /etc/hosts file quickly grew too large to be manageable.
See historical document below.
TEXT hosts.hst from textfiles.com
Internet users switched to DNS.
Recently, for various reasons, i became interested in avoiding DNS,
so i researched using an /etc/hosts file. I found one on
gopher.quest, but unfortunately it is incomplete.
DIR hosts.txt at gopher.quest
I noticed that the Quarry gopher search engine contains a little over
300 live gopher hosts. I wrote the author about it, and he generously
created an /etc/hosts file from the Quarry database, linked below.
TEXT hosts.txt at gopher.icu
This is updated daily. On a linux system, you could replace
/etc/hosts with this file and assuming a sane C library, it should
"just work." It will resolve those names without DNS.
FreeDOS is slightly more complicated. Suppose the file is in
C:/etc/hosts.txt and will be used by mTCP and WATTCP.
For mTCP, edit mtcp.cfg, add:
HOSTSFILE C:/etc/hosts.txt
For WATTCP, edit wattcp.cfg, add:
hosts = C:/etc/hosts.txt
Now FreeDOS programs should resolve those names without DNS.
This covers gopher hosts. I would also like to include FTP and web
hosts referenced by my gopher maps. Below is a script i wrote to
mirror gopher content.
TEXT gopher-get.tcl
BIN tclcurl.tgz SlackBuild
Using this script, i mirrored my gopher content.
$ mkdir gopherhole
$ cd gopherhole
$ tclsh gopher-get.tcl --binary-skip --images-skip \
--ignore gopher://tilde.pink/1/~bencollver/dir \
--ignore gopher://tilde.pink/1/~bencollver/dict \
--ignore gopher://tilde.pink/1/~bencollver/gamefaqs \
--ignore gopher://tilde.pink/1/~bencollver/ia \
--ignore gopher://tilde.pink/1/~bencollver/recipes \
gopher://tilde.pink/1/~bencollver
This took a while, so i let it run and came back later. This created
two directories.
* tilde.pink/0 (Plain text documents)
* tilde.pink/1 (Gopher maps)
I scraped the host names from these directories as follows.
$ find tilde.pink/1 -type f -exec \
grep -e URL:ftp: -e URL:http: -e URL:https: {} \; |\
sed -e 's,.*URL:[^:]*://,,' -e 's,/.*,,' >names-gopher.txt
$ find tilde.pink/0 -type f -exec \
awk -f setext-scrape-links.awk {} \; >names-setext.txt
$ cat names-gopher.txt names-setext.txt | sort |\
uniq >names-extra.txt
$ wc -l names-extra.txt
392 names-extra.txt
TEXT setext-scrape-links.awk
This shows that i reference approximately 400 non-gopher names. Now i
would like to resolve these to IP addresses. I wrote a short AWK
script to do this. I ignored unkown names because of old documents
that reference defunct names.
$ awk -f resolve.awk names-extra.txt >hosts-extra.txt
$ wc -l hosts-extra.txt
612 hosts-extra.txt
TEXT resolve.awk
Because DNS can resolve a name to multiple IP addresses, this list
grew from 392 to 612 lines. I appended this list to the Quarry hosts
file like so.
$ curl -o hosts-icu.txt gopher://gopher.icu/0/files/hosts.txt
$ cat hosts-icu.txt hosts-extra.txt >hosts-all.txt
$ wc -l hosts-all.txt
1058
For FreeDOS, filter out ":" to exclude IPv6 addresses. Also, mTCP
has an 80 character line length limit including EOL characters.
$ grep -v : hosts-all.txt |\
awk 'length($0) < 78 {print}' >hosts-dos.txt
Now i can copy hosts-all.txt to /etc/hosts
or copy hosts-dos.txt to C:/etc/hosts.txt
and i am good to go.
p.s.
The FreeDOS bundled mTCP has a bug. If you specify HOSTSFILE but no
NAMESERVER, then name resolution will always fail. You can work
around this by making sure mtcp.cfg has a NAMESERVER line. If you
don't use DNS at all, it can be:
NAMESERVER 127.0.0.1
tags: bencollver,retrocomputing,technical,unix
# Tags
DIR bencollver
DIR retrocomputing
DIR technical
DIR unix