2025-08-03 GoToSocial and the Butlerian Jihad
=============================================

I run a single user instance for the fediverse using GoToSocial and
since I'm blocking thousands of IP address ranges -- mostly from
companies renting out bandwidth and computing power to web scrapers --
I'm also blocking my instance from seeing a lot of the fediverse.
That's not good.

The solution is to continue blocking the big cloud providers like
Microsoft, Google, Amazon, Hentzner, OVH, Alibaba and whatever else
they are called. But if I'm following somebody from a fedi instance
hosted by these big cloud providers, I should put the IP numbers
associated with just those domain names onto a separate allow list.

Instead of using the Mastodon client protocol to get the list of
people I'm following, I can just access the database directly.

Here's the script that adds IP addresses:

    #!/usr/bin/fish
    # Redirect the output of this script into /etc/butlerian-jihad/gotosocial.sh
    echo "#" (date --iso)
    set domains (sqlite3 /home/gotosocial/sqlite.db '
    select distinct b.domain from follows as f
    join accounts as a on f.account_id = a.id
    join accounts as b on f.target_account_id = b.id
    where a.username = "alex"
    and b.domain is not null
    union
    select distinct b.domain from follow_requests as f
    join accounts as a on f.account_id = a.id
    join accounts as b on f.target_account_id = b.id
    where a.username = "alex"
    and b.domain is not null
    order by 1')
    set i 1
    set m (count $domains)
    for domain in $domains
        set ipv6 (host -t AAAA $domain | awk '/has IPv6 address/ { print $5}')
        set ipv4 (host -t A $domain | awk '/has address/ { print $4}')
        echo "$i/$m $domain:" $ipv4 $ipv6 >&2
        set i (math 1 + $i)
        for ip in $ipv4
            echo "ipset add gotosocial $ip # $domain"
        end
        for ip in $ipv6
            echo "ipset add gotosocial6 $ip # $domain"
        end
    end

This prints lines such as the following to STDERR:

    157/244 republik.ch: 172.67.74.47 104.26.5.143 104.26.4.143 2606:4700:20::681a:48f 2606:4700:20::ac43:4a2f 2606:4700:20::681a:58f

And it prints lines such as the following to STDOUT:

    ipset add gotosocial 172.67.74.47 # republik.ch
    ipset add gotosocial 104.26.5.143 # republik.ch
    ipset add gotosocial 104.26.4.143 # republik.ch
    ipset add gotosocial6 2606:4700:20::681a:48f # republik.ch
    ipset add gotosocial6 2606:4700:20::ac43:4a2f # republik.ch
    ipset add gotosocial6 2606:4700:20::681a:58f # republik.ch

I redirect STDOUT to a file called gotosocial.sh and then I run this
script, which creates to IP sets called gotosocial and gotosocial6:

    #!/usr/bin/fish
    # https://www.digitalocean.com/community/tutorials/iptables-essentials-common-firewall-rules-and-commands
    
    ipset create gotosocial hash:ip
    ipset create gotosocial6 hash:ip family inet6
    
    # this set is inserted at the beginning
    iptables -I INPUT -m set --match-set gotosocial src -j ACCEPT
    ip6tables -I INPUT -m set --match-set gotosocial6 src -j ACCEPT
    
    # recreate using: gotosocial-follower-ips > gotosocial.sh
    source gotosocial.sh
    
    netfilter-persistent save
    
    # Find the number of the rule (hopefully it is number one)
    #   iptables --list INPUT
    #   ip6tables --list INPUT
    # To delete, verify the number of the rule:
    #   iptables --list INPUT 1
    #   ip6tables --list INPUT 1
    # Now delete it the rule:
    #   iptables --delete INPUT 1
    #   ip6tables --delete INPUT 1
    # Now you can destroy the two ipsets, too
    #   ipset destroy gotosocial
    #   ipset destroy gotosocial6
    # And now you can rerun this script!

If you run it, you'll get warnings about duplicate entries, I'm sure:

    ipset v7.17: Element cannot be added to the set: it's already added

In my case, for example, I have both of these entries and I didn't
bother to fix this:

    ipset add gotosocial 151.101.129.91 # mastodon.online
    ipset add gotosocial 151.101.129.91 # mastodon.social

There are plenty more, of course.

In any case, you should now have an allow list that prevents these IP
addresses from getting blocked by fail2ban because the gotosocial
rule comes first:

    # iptables --list INPUT
    Chain INPUT (policy ACCEPT)
    target     prot opt source               destination         
    ACCEPT     all  --  anywhere             anywhere             match-set gotosocial src
    ACCEPT     all  --  anywhere             anywhere             match-set allowlist src
    f2b-alex-apache  tcp  --  anywhere             anywhere             multiport dports http,https
    f2b-butlerian-jihad  tcp  --  anywhere             anywhere             multiport dports 0:65535
    f2b-butlerian-jihad-week  tcp  --  anywhere             anywhere             multiport dports 0:65535
    f2b-alex-bots  tcp  --  anywhere             anywhere             multiport dports http,https
    f2b-recidive  tcp  --  anywhere             anywhere            
    DROP       all  --  anywhere             anywhere             match-set banlist src

To delete and recreate the lists, I use the following:

    #!/usr/bin/fish
    ipset flush gotosocial
    ipset flush gotosocial6
    source gotosocial.sh
    netfilter-persistent save

#GoToSocial #Butlerian_Jihad