publishing the blocklist: an API for what the signup bot has blocked

Years ago I worked in the email industry, and one of the biggest challenges was spam. Less about spam signups, and more about receiving spam from bad senders, but the idea applies to mastodon signups – there are some bad IP ranges and while it’s good to keep track for our own instances, mastodon could really benefit from sharing block lists.

I’ve decided to share my instance’s IP block list, and want to think more about how to make these things a little more automated (ie, a DNS based lookup at signup).

The welcome-bot/abuse-bot stack has been quietly auto-blocking bad signups for a couple of months now — mostly datacenter and VPN ranges. The active blocks can be accessed via

https://hooks.yttrx.com/api/blocks/ip-ranges

JSON by default. Add ?format=txt to either of the per-list endpoints and you get a plain one-per-line list with a # comment header, which is the form blocklists actually get traded in:

curl -s 'https://hooks.yttrx.com/api/blocks/ip-ranges?format=txt' | grep -v '^#'

At the time of writing that’s 121 networks and 106 domains.

What’s deliberately not in it #

Any of the reasoning behind the block. I’ve posted already about using IP reputation lists to lookup the signup IP, and if the network falls into particular classes (bot, vpn, etc) then I block the entire CIDR for a period of time. I don’t want to republish any of that specific data, just the IP ranges.

Mastodon can’t consume this directly, and that annoys me #

Initially I assumed there’d be a DNSBL style lookup that could be leveraged, but after some grepping around I couldn’t find anything outside of the mx lookup of blocked addresses. This feels like a major gap, but getting that upstreamed will be a bunch of work that I currently don’t have the time to manage, so I’ll add it to my todo list. This means that the list needs to be imported; I’ve got a little script for that to make things go faster (or until there’s time to get this upstreamed).

./bin/yttrx-blocklist-import --instance https://your.instance --token "$TOKEN"
./bin/yttrx-blocklist-import --instance https://your.instance --token "$TOKEN" --apply

If you do import it, read this bit. The IP entries are whole networks — the ASN route a flagged address belongs to, not the address itself. That’s right for yttrx because a flagged signup here is nearly always datacenter, VPN or proxy space, where the next attempt comes from a neighbouring address and blocking a single /32 accomplishes approximately nothing. It might be badly wrong for you. If you have real users behind a VPN, a blanket network block is going to catch them.

So the importer defaults to sign_up_requires_approval — flagged signups land in your moderation queue — rather than the sign_up_block hard rejection yttrx runs on itself. You can opt into the harder setting with --ip-severity sign_up_block, but that should be a decision you make about your own traffic, not one you inherit from mine.

Code, as ever: git.blairhaus.net/pmb/yttrx-welcomebot . If you import it and something’s wrong, or if a range in there is yours and shouldn’t be, tell me. I prefer email for this stuff.