Site Activity
-
wpconvert replied to the topic Suggestion: Restrict additional files in DENY BROWSER ACCESS TO THESE FILES in the forum BulletProof Security Pro 1 day, 8 hours ago
Ahh I see – ok. I just automatically associated it with a bot probe when I matched up the server log entries with php errors I was getting on a number of sites – as I haven’t seen this error before.
I’m quite confident that it wouldn’t be anything related to server load – I have very decent resources.
There is no harm in adding wp-settings.php…[Read more]
-
AITpro Admin replied to the topic Suggestion: Restrict additional files in DENY BROWSER ACCESS TO THESE FILES in the forum BulletProof Security Pro 1 day, 10 hours ago
Yep, I have seen that error before when updating WordPress. It is caused by a delay/latency in the WordPress update process. What the error means is that the constant “ABSPATH” is not seen as loaded yet. This error can also be caused by intermittent server load issues. ie if your server is briefly overloaded then dependent files may not load in…[Read more]
-
wpconvert started the topic Suggestion: Restrict additional files in DENY BROWSER ACCESS TO THESE FILES in the forum BulletProof Security Pro 1 day, 14 hours ago
Hi.
I’ve been getting a number of these kinds of error in my server error log recently:
[18-Jul-2026 20:36:09 UTC] PHP Fatal error: Uncaught Error: Undefined constant “ABSPATH” in /home/mydomain/public_html/wp-settings.php:34 Stack trace: #0 {main} thrown in /home/mydomain/public_html/wp-settings.php on line 34This has occurred after updating…[Read more]
-
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days ago
Thanks!
-
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days ago
Most bad Bots are just annoying and are not malicious. Bad Bots were a bigger issue many years ago because they caused excessive server load. These days the server load is insignificant because PHP, web servers, etc. have improved so much. So since you are blocking Bots in general it is important to allow the specific good Bots that you want to…[Read more]
-
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 2 hours ago
Thanks a lot for your replies and insight! it seems that my effort to allow SEO crawlers, like AhrefsBot or SemrushBot, even OAI-SearchBot (which is OpenAI indexer, not for training) , are not something other webmasters do? I say this because allowing them seems to have some risks that are difficult to mitigage. I was under the impression that…[Read more]
-
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 2 hours ago
My honest opinion – That would take significant effort for very little in return. Or in other words, not really worth the effort.
In order for your checks to work you would have to grab a list of valid IP addresses for Meta, put them into an array and check against that array.
Something like this generally…
If User Agent = X and IP…[Read more]
-
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago
Got it. So, I will rephrase my question. If I get a visit like this
[403 GET Request: Juli 21, 2026 – 2:23 p.m.]
BPS Pro: 17.6
WP: 7.0.2
Event Code: BFHS – Blocked/Forbidden Hacker or Spammer
Solution: N/A – Hacker/Spammer Blocked/Forbidden
REMOTE_ADDR: 57.141.0.17
Host Name: 57.141.0.17
SERVER_PROTOCOL:…[Read more] -
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago
https://stackoverflow.com/questions/45836233/ptr-record-is-confusing
Yes, it is relatively simple to create reverse DNS checking code using PHP code, but here are the pitfalls. The IP address you are using to check with could belong to an innocent web host who is hosting a malicious person or what is more common is the IP address you are using to…[Read more]
-
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago
thanks for the htaccess file, I will take a look at it
-
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago
Thanks for your reply. Actually, what I want to achieve is only a reverse dns, to make sure that e.g., when a meta crawler HEAD or GET from 57.141.0.17 arrives, something checks the IP belongs to Meta and if not, drop it. There is a plugin that does that, BotBlocker Security, but I’m reluctant to have it along BPS PRO.
-
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago
Jeff Starr at Perishable Press created an extensive htaccess file here > https://perishablepress.com/6g/
Warning: You will need to test everything on your website to make sure nothing important is being blocked.
-
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago
It’s very quick and simple to change the User Agent. All browsers have tools that let you do that. When someone is automating Bots from a server the Bot code is designed to automatically change User Agents if the Bot is blocked. If blocked, then try X User Agent. That leaves you with IP blocking, which is a waste of time since hackers and spammers…[Read more]
-
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 4 hours ago
Thanks..Now, a (somehow) related question, is there a way with BPS pro to make sure a hacker can not impersonate a crawler, by a reserve DNS or PHP-level check?
-
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 4 hours ago
The HEAD Request htaccess code is not website security code. It’s nuisance protection against Bots. Not sure why the code I posted is not working. Your code looks fine. That could be because I added the ^ character, which means start checking here, but did not add the closing $ character.
RewriteCond %{HTTP_USER_AGENT} !^UptimeRobot.*$…[Read more]
-
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 5 hours ago
I implemented this change, allow GET and HEAD for crawlers, allow HEAD for uptimerobot, disallow everything else, and it seems to be working fine. Please let me know if this ok, security-wise.
# CRAWLER USER-AGENT WHITELIST — bypass all further BPS rewrite rules for approved bots
RewriteEngine On
RewriteCond %{REQUEST_METHOD} ^(GET|HEAD)$ […[Read more] -
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 5 hours ago
Hi..I implemented the code you sent me, and now uptimerobot is not working, nor the crawlers
[405 HEAD Request: July 21, 2026 – 2:06 pm]
BPS Pro: 17.6
WP: 7.0.2
Event Code: BFHS-HEAD – HEAD Request Blocked
Solution: https://forum.ait-pro.com/forums/topic/security-log-event-codes/
REMOTE_ADDR: 52.22.236.30
Host Name:…[Read more] -
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 6 hours ago
Since you have modified your custom htaccess code to use the BPS GET Request error template then that is why it is being logged as a GET Request. At least as far as the template label goes. The BPS 405 error template has this template label: 405 HEAD Request:
RewriteCond %{REQUEST_METHOD} ^HEAD [NC]
RewriteRule ^ – [F] -
AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 6 hours ago
All Bots make HEAD Requests first and then may use a GET Request after the HEAD Request. Sometimes HEAD Request are seen as GET Requests.
Note: SemrushBot.* would match: SemrushBot/7~bl; +http://www.semrush.com/bot.html)
Try this code:
# REQUEST METHODS FILTERED
# If you want to allow HEAD Requests use BPS Custom Code and copy
# this…[Read more] -
Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 6 hours ago
Thanks for your prompt reply. Actually, I was using the uptimerobot tweak already, so this is what I have now in place:
# REQUEST METHODS FILTERED
RewriteEngine On
# 1. ALLOW UptimeRobot and approved crawlers to make HEAD requests
# If HEAD *AND* UA matches an allowed bot, stop processing here and let the request through normally.
RewriteCond…[Read more] - Load More



