%s

2023-10-15 4chan client shell script I wrote this shell script a while back because I wanted to browse the Lisp and Emacs general thread on 4chans /g/ board[1] in Emacs. I thought about writing it in Elisp but handling Json data is a real pain. So instead I use curl[2] to request a thread via the 4chan API[3], then pipe it to jq[4] to process the result, then AWK[5] to work its black magic to construct a very simplified html version of the thread which I can finally view with Eww[6] in Emacs. Looking at this code a few months later it kinda looks like the work of a madman, though. Features ~~~~~~~~ - produces a simple html file ideal for text browsers - shows replies for every post (see screenshot) - images are linked Drawbacks ~~~~~~~~~ - if no thread number is provided it shows a list of threads from the catalogue but ONLY those with a proper title - the /g/ board is hard-coded (which can be easily changed of course) Screenshot ~~~~~~~~~~ (lol) - Lisp General (M-x eshell Edition): file:///tmp/thread.html [IMG] 96623294 Do 12. Okt 17:29:30 CEST 2023 How can I mentally think of C-x? It seems to do too much. Replies: >>96623403 >>96623705 [IMG] 96623403 Do 12. Okt 17:34:58 CEST 2023 >>96623294 It's a sequence of keys, the control key and the x key Replies: >>96623624 [IMG] 96623624 Do 12. Okt 17:45:21 CEST 2023 >>96623403 I know, but what do I call it? Replies: >>96623807 -UUU: @%*- F2 *eww* 9% (165,0) (eww) ----------- Code ~~~~ #!/bin/sh # if no thread number is provided if [ $# -lt 1 ]; then # print catalog, ONLY THREADS WITH TITLE (because I only need /lol/ # and printing the OP would be too much clutter, html and all) # https://github.com/4chan/4chan-API/blob/master/pages/Catalog.md curl -s https://a.4cdn.org/g/catalog.json | \ jq -r '.[].threads | map([.no, .sub] | join(" ")) | join("\n")' | \ awk '$2' exit 1 fi # pipeline: curl (thread.json) -> jq (thread.csv) -> awk (thread.html) # https://github.com/4chan/4chan-API/blob/master/pages/Threads.md curl -s https://a.4cdn.org/g/thread/$1.json > /tmp/thread.json # extract title jq -r '.[][0].sub' /tmp/thread.json > /tmp/thread.csv # extract thread-number, unix time stamp, comment, # unix timestamp + microtime (=filename), file extension jq -r '.posts | map([.no, .time, .com, .tim, .ext] | join("\t")) | join("\n")' \ /tmp/thread.json >> /tmp/thread.csv # black awk magic awk ' BEGIN { FS="\t"; } # first loop over file to get references NR==FNR && match ($0, />>[0-9]*/) { # e.g. match = >>91962953, (> is html code for >) split(substr($0, RSTART, RLENGTH), to_arr, ";"); # e.g. x = 91962953 (= referenced post), no capture groups in mawk x=to_arr[3]; # generate new html link new_link = sprintf(">>%s", $1 ,$1); # replies[referenced_post] += referring_post (string concat) replies[x] = sprintf("%s %s", replies[x] , new_link); } # second loop, write html NR!=FNR && FNR==1 { printf "%s", $0; } NR!=FNR && FNR >1 { print ""; # insert image link or placeholder print ""; # insert post id, creation date of post in local time and comment sprintf("date -d @%s", $2) | getline local_time printf ""; # insert replies to this post if (replies[$1]) { print ""; } print "

"; if ($4) printf "[IMG]", $4, $5; else print "[IMG]"; print "	"; printf "%s %s %s", $1, $1, local_time, $3; print "
	Replies: "; print replies[$1]; print "

"; } END{ print ""; }' /tmp/thread.csv /tmp/thread.csv > /tmp/thread.html # open in *eww* emacsclient --eval '(eww-open-file "/tmp/thread.html")' Footnotes ~~~~~~~~~ [1] https://boards.4channel.org/g/ [2] https://curl.se/ [3] https://github.com/4chan/4chan-API/blob/master/pages/Threads.md [4] https://github.com/jqlang/jq [5] https://en.wikipedia.org/wiki/AWK [6] https://www.gnu.org/software/emacs/manual/html_mono/eww.html