2024-12-17 - No DNS, No Problem =============================== The early Internet used /etc/hosts files to resolve names. These were distributed on FTP sites. You can see a 1992 example below. host.txt from textfiles.com The /etc/hosts file quickly grew too large to be manageable. See historical document below. hosts.hst from textfiles.com Internet users switched to DNS. Recently, for various reasons, i became interested in avoiding DNS, so i researched using an /etc/hosts file. I found one on gopher.quest, but unfortunately it is incomplete. hosts.txt at gopher.quest I noticed that the Quarry gopher search engine contains a little over 300 live gopher hosts. I wrote the author about it, and he generously created an /etc/hosts file from the Quarry database, linked below. hosts.txt at gopher.icu This is updated daily. On a linux system, you could replace /etc/hosts with this file and assuming a sane C library, it should "just work." It will resolve those names without DNS. FreeDOS is slightly more complicated. Suppose the file is in C:/etc/hosts.txt and will be used by mTCP and WATTCP. For mTCP, edit mtcp.cfg, add: HOSTSFILE C:/etc/hosts.txt For WATTCP, edit wattcp.cfg, add: hosts = C:/etc/hosts.txt Now FreeDOS programs should resolve those names without DNS. This covers gopher hosts. I would also like to include FTP and web hosts referenced by my gopher maps. Below is a script i wrote to mirror gopher content. gopher-get.tcl tclcurl.tgz SlackBuild Using this script, i mirrored my gopher content. $ mkdir gopherhole $ cd gopherhole $ tclsh gopher-get.tcl --binary-skip --images-skip \ --ignore gopher://tilde.pink/1/~bencollver/dir \ --ignore gopher://tilde.pink/1/~bencollver/dict \ --ignore gopher://tilde.pink/1/~bencollver/gamefaqs \ --ignore gopher://tilde.pink/1/~bencollver/recipes \ gopher://tilde.pink/1/~bencollver This took a while, so i let it run and came back later. This created two directories. * tilde.pink/0 (Plain text documents) * tilde.pink/1 (Gopher maps) I scraped the host names from these directories as follows. $ find tilde.pink/1 -type f -exec \ grep -e URL:ftp: -e URL:http: -e URL:https: {} \; |\ sed -e 's,.*URL:[^:]*://,,' -e 's,/.*,,' >names-gopher.txt $ find tilde.pink/0 -type f -exec \ awk -f setext-scrape-links.awk {} \; >names-setext.txt $ cat names-gopher.txt names-setext.txt | sort |\ uniq >names-extra.txt $ wc -l names-extra.txt 392 names-extra.txt setext-scrape-links.awk This shows that i reference approximately 400 non-gopher names. Now i would like to resolve these to IP addresses. I wrote a short AWK script to do this. I ignored unkown names because of old documents that reference defunct names. $ awk -f resolve.awk names-extra.txt >hosts-extra.txt $ wc -l hosts-extra.txt 612 hosts-extra.txt resolve.awk Because DNS can resolve a name to multiple IP addresses, this list grew from 392 to 612 lines. I appended this list to the Quarry hosts file like so. $ curl -o hosts-icu.txt gopher://gopher.icu/0/files/hosts.txt $ cat hosts-icu.txt hosts-extra.txt >hosts-all.txt $ wc -l hosts-all.txt 1058 For FreeDOS, filter out ":" to exclude IPv6 addresses. Also, mTCP has an 80 character line length limit including EOL characters. $ grep -v : hosts-all.txt |\ awk 'length($0) < 78 {print}' >hosts-dos.txt Now i can copy hosts-all.txt to /etc/hosts or copy hosts-dos.txt to C:/etc/hosts.txt and i am good to go. p.s. The FreeDOS bundled mTCP has a bug. If you specify HOSTSFILE but no NAMESERVER, then name resolution will always fail. You can work around this by making sure mtcp.cfg has a NAMESERVER line. If you don't use DNS at all, it can be: NAMESERVER 127.0.0.1 tags: bencollver,retrocomputing,technical,unix Tags ==== bencollver retrocomputing technical unix