Subj : Web server and BOT's
To   : Richard Vonzel
From : nelgin
Date : Wed Jun 25 2025 23:26:42

  Re: Web server and BOT's
  By: Richard Vonzel to All on Wed Jun 25 2025 04:25:24

 > I turned on the Synchronet Web server, works great, hsad my website running.
 > Had my BBS's files available for access thru it. Yesterday I guess a BOT got
 > on my site and jst downloaded all the file I had available. Here's the log
 > entry for this,

I have found gptbot ignores a global disallow rule in robots.txt

User-agent: *
Disallow: /

But you could try adding

User-agent: gptbot
Disallow :/

It could take a few days but you'll see when they start to pickup your robots.txt file.

Another option would be to just block the ip ranges altogether.

Another option would be to block the published subnets. I have these blocked using ipsets

4.227.36.0/25
20.171.207.0/24
52.230.152.0/24
172.182.204.0/24
20.171.206.0/24
172.182.214.0/24
20.125.66.80/28
172.182.215.0/24

That seems to be the easiest way and just don't deal with them at all.

---
 þ Synchronet þ End Of The Line BBS - endofthelinebbs.com
 * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

.