2025-07-09 systemd timers for the Butlerian Jihad
=================================================

I was reading a post about the good parts of systemd (systemd has
been a complete, utter, unmitigated success) and started wondering
about the use of timers instead of cron jobs.

The benefits:

* each job is a complete script

* each job can be sandboxed

* each job has output in the log instead of mailing it to me

And if every job is a complete script, then I can also add more stuff,
like lowering the limits if load starts to overshoot again.

So here we go.

I have three jobs:

* watch-active-autonomous-systems

* watch-expensive-end-points

* watch-nobots

Each of them gets a service and a timer.

Since I wanted the service to protect a good part of the system, I had
to move things around a bit.

asncounter


I had to install asncounter in /usr/local. Since I already had a
copy of it all I just moved things around, but things to work out next
time, my root user has PIPX_HOME set to /usr/local/pipx. This is
very messy. I hope that asncounter makes it into Debian, soon. ❤️

So:

    mv ~/.local/pipx /usr/local

I also edited the first line of asncounter to read as follows:

    #!/usr/local/pipx/venvs/asncounter/bin/python

/etc/butlerian-jihad


I created this directory for the scripts and its dependencies.

* 2h-access-log

* asn-networks

I also create a data directory here for the pyasn data. This is not
the default location so we must always pass --cache-directory
/etc/butlerian-jihad/data when running asncounter.

scripts
-------

The three scripts all exclude some more IP numbers in their call to
2h-access-log (my home IP numbers, my server's IP numbers) so I don't
accidentally ban myself, as well as the social subdomain which is
where my fedi instance is.

watch-active-autonomous-systems


    #!/usr/bin/sh
    export PATH=/etc/butlerian-jihad:/usr/local/pipx/venvs/asncounter/bin:$PATH
    2h-access-log !^social $MY_IPS \
        | awk '{print $2}' \
        | asncounter --top 50 --no-prefixes --cache-directory /etc/butlerian-jihad/data 2>/dev/null \
        | awk '/^[0-9]/ && $1>500 { print $3 }' \
        | ifne xargs /etc/butlerian-jihad/asn-networks \
        | ifne xargs fail2ban-client set butlerian-jihad banip

watch-expensive-end-points


    #!/usr/bin/sh
    export PATH=/etc/butlerian-jihad:/usr/local/pipx/venvs/asncounter/bin:$PATH
    2h-access-log !^social $MY_IPS \
        | egrep '\baction=(rss|rc)\&|\bsearch=' \
        | awk '{print $2}' \
        | asncounter --top 50 --no-prefixes --cache-directory /etc/butlerian-jihad/data 2>/dev/null \
        | awk '/^[0-9]/ && $1>10 { print $3 }' \
        | ifne xargs /root/bin/asn-networks \
        | ifne xargs fail2ban-client set butlerian-jihad banip

watch-nobots


    #!/usr/bin/sh
    export PATH=/etc/butlerian-jihad:/usr/local/pipx/venvs/asncounter/bin:$PATH
    2h-access-log !^social $MY_IPS \
        | grep "GET /nobots" \
        | awk '{print $2}' \
        | asncounter --top 50 --no-prefixes --cache-directory /etc/butlerian-jihad/data 2>/dev/null \
        | awk '/^[0-9]/ && $1>30 { print $3 }' \
        | ifne xargs /root/bin/asn-networks \
        | ifne xargs fail2ban-client set butlerian-jihad banip

*.service


Each script gets a service file. I'm only going to post
watch-active-autonomous-systems.service. The only thing that changes
from service to service is the Description and the ExecStart naming
the script to run.

    [Unit]
    Description=Watch active autonomous systems
    RequiresMountsFor=/var/log
    ConditionACPower=true
    
    [Service]
    Type=oneshot
    ExecStart=/etc/butlerian-jihad/watch-active-autonomous-systems
    
    # Priority has to be higher than the regular web services so that banning can still happen.
    # See systemd.exec(5) for more.
    Nice=9
    IOSchedulingClass=best-effort
    IOSchedulingPriority=3
    
    # asncounter needs to download new route view data
    # PrivateNetwork=true
    
    # asncounter needs to save the data somewhere
    ReadWritePaths=/etc/butlerian-jihad/data
    
    LockPersonality=true
    MemoryDenyWriteExecute=true
    NoNewPrivileges=true
    PrivateDevices=true
    PrivateTmp=true
    ProtectClock=true
    ProtectControlGroups=true
    ProtectHome=true
    ProtectHostname=true
    ProtectKernelLogs=true
    ProtectKernelModules=true
    ProtectKernelTunables=true
    ProtectSystem=full
    RestrictNamespaces=true
    RestrictRealtime=true
    RestrictSUIDSGID=true

*.timer


Each script gets a timer file. I'm only going to post one of them. The
only thing that changes from service to service is the Description as
by default the timer applies for the service with the same name. Win!

I'm adding a RandomizedDelaySec of two minutes, hoping that over time
the three timers will start to diverge.

    [Unit]
    Description=Watch active autonomous systems
    
    [Timer]
    OnCalendar=*:00,10,20,30,40,50:00
    RandomizedDelaySec=120
    
    [Install]
    WantedBy=timers.target

systemd cheat-sheet
-------------------

    # install service or timer (only once)
    systemctl enable --now ./the-job.service
    systemctl enable --now ./the-job.timer
    # check how it went
    systemctl status the-job.service
    systemctl status the-job.timer
    # run it again
    systemctl start the-job.service
    systemctl start the-job.timer
    # check the logs (use --follow for watching)
    journalctl --unit the-job.service
    journalctl --unit the-job.timer

#Administration #Butlerian_Jihad #systemd