How to block empty user-agent string to stop bots

[Frank]

Hi,

I need help blocking the relentless daily crawling from bots in this range  CIDR: 47.80.0.0/13, 47.76.0.0/14, 47.74.0.0/15 (47.80.0.0 – 47.95.255.255).  There is no user-agent.  From Singapore,  Alibaba Cloud LLC.

I have blocked country blocking, IP range, CIDR’s, some individual IPs to no effect.  Rate limited to almost nothing in Wordfence but the bots over-ride everything.  They are ‘blocked’ in the sense from the content but are still trawling and using bandwidth etc.

I have the following at the top of .htaccess but that did nothing.

# Block empty user-agent string
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteRule ^ - [F,L]

Is there some other custom code that might stop these bots crawling my site? Or something else I can do?

Thank you for your time,

Kind regards

[AITpro Admin]

Trying to block Bots is futile even when using CIDR or country blocking. Bots should not cause any significant resource and bandwidth usage.  Why do you feel that Bots are causing significant resource and bandwidth usage.

[Author]

Posts