BulletProof Security Forum

Los Angeles, California

Follow @BPSPro

Site Activity

  • All Members 21,318
  • RSS
  • Profile picture of wpconvert

    wpconvert replied to the topic Suggestion: Restrict additional files in DENY BROWSER ACCESS TO THESE FILES in the forum BulletProof Security Pro 1 day, 8 hours ago

    Ahh I see – ok. I just automatically associated it with a bot probe when I matched up the server log entries with php errors I was getting on a number of sites –  as I haven’t seen this error before.

    I’m quite confident that it wouldn’t be anything related to server load – I have very decent resources.

    There is no harm in adding wp-settings.php…[Read more]

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Suggestion: Restrict additional files in DENY BROWSER ACCESS TO THESE FILES in the forum BulletProof Security Pro 1 day, 10 hours ago

    Yep, I have seen that error before when updating WordPress. It is caused by a delay/latency in the WordPress update process. What the error means is that the constant “ABSPATH” is not seen as loaded yet. This error can also be caused by intermittent server load issues. ie if your server is briefly overloaded then dependent files may not load in…[Read more]

  • Profile picture of wpconvert

    wpconvert started the topic Suggestion: Restrict additional files in DENY BROWSER ACCESS TO THESE FILES in the forum BulletProof Security Pro 1 day, 14 hours ago

    Hi.
    I’ve been getting a number of these kinds of error in my server error log recently:
    [18-Jul-2026 20:36:09 UTC] PHP Fatal error: Uncaught Error: Undefined constant “ABSPATH” in /home/mydomain/public_html/wp-settings.php:34 Stack trace: #0 {main} thrown in /home/mydomain/public_html/wp-settings.php on line 34

    This has occurred after updating…[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days ago

    Thanks!

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days ago

    Most bad Bots are just annoying and are not malicious. Bad Bots were a bigger issue many years ago because they caused excessive server load. These days the server load is insignificant because PHP, web servers, etc. have improved so much. So since you are blocking Bots in general it is important to allow the specific good Bots that you want to…[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 2 hours ago

    Thanks a lot for your replies and insight! it seems that my effort to allow SEO crawlers, like AhrefsBot or SemrushBot, even OAI-SearchBot (which is OpenAI indexer, not for training) , are not something other webmasters do? I say this because allowing them seems to have some risks that are difficult to mitigage. I was under the impression that…[Read more]

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 2 hours ago

    My honest opinion – That would take significant effort for very little in return. Or in other words, not really worth the effort.

    In order for your checks to work you would have to grab a list of valid IP addresses for Meta, put them into an array and check against that array.

    Something like this generally…

    If User Agent = X and IP…[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago

    Got it. So, I will rephrase my question. If I get a visit like this

    [403 GET Request: Juli 21, 2026 – 2:23 p.m.]
    BPS Pro: 17.6
    WP: 7.0.2
    Event Code: BFHS – Blocked/Forbidden Hacker or Spammer
    Solution: N/A – Hacker/Spammer Blocked/Forbidden
    REMOTE_ADDR: 57.141.0.17
    Host Name: 57.141.0.17
    SERVER_PROTOCOL:…[Read more]

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago

    https://stackoverflow.com/questions/45836233/ptr-record-is-confusing

    Yes, it is relatively simple to create reverse DNS checking code using PHP code, but here are the pitfalls. The IP address you are using to check with could belong to an innocent web host who is hosting a malicious person or what is more common is the IP address you are using to…[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago

    thanks for the htaccess file, I will take a look at it

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago

    Thanks for your reply. Actually, what I want to achieve is only a reverse dns, to make sure that e.g., when a meta crawler HEAD or GET from 57.141.0.17 arrives, something checks the IP belongs to Meta and if not, drop it. There is a plugin that does that, BotBlocker Security, but I’m reluctant to have it along BPS PRO.

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago

    Jeff Starr at Perishable Press created an extensive htaccess file here > https://perishablepress.com/6g/

    Warning: You will need to test everything on your website to make sure nothing important is being blocked.

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 3 hours ago

    It’s very quick and simple to change the User Agent. All browsers have tools that let you do that. When someone is automating Bots from a server the Bot code is designed to automatically change User Agents if the Bot is blocked. If blocked, then try X User Agent. That leaves you with IP blocking, which is a waste of time since hackers and spammers…[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 4 hours ago

    Thanks..Now, a (somehow) related question, is there a way with BPS pro to make sure a hacker can not impersonate a crawler, by a reserve DNS or PHP-level check?

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 4 hours ago

    The HEAD Request htaccess code is not website security code. It’s nuisance protection against Bots. Not sure why the code I posted is not working. Your code looks fine. That could be because I added the ^ character, which means start checking here, but did not add the closing $ character.

    RewriteCond %{HTTP_USER_AGENT} !^UptimeRobot.*$…[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 5 hours ago

    I implemented this change, allow GET and HEAD for crawlers, allow HEAD for uptimerobot, disallow everything else, and it seems to be working fine. Please let me know if this ok, security-wise.

    # CRAWLER USER-AGENT WHITELIST — bypass all further BPS rewrite rules for approved bots
    RewriteEngine On
    RewriteCond %{REQUEST_METHOD} ^(GET|HEAD)$ […[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 5 hours ago

    Hi..I implemented the code you sent me, and now uptimerobot is not working, nor the crawlers

    [405 HEAD Request: July 21, 2026 – 2:06 pm]
    BPS Pro: 17.6
    WP: 7.0.2
    Event Code: BFHS-HEAD – HEAD Request Blocked
    Solution: https://forum.ait-pro.com/forums/topic/security-log-event-codes/
    REMOTE_ADDR: 52.22.236.30
    Host Name:…[Read more]

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 6 hours ago

    Since you have modified your custom htaccess code to use the BPS GET Request error template then that is why it is being logged as a GET Request. At least as far as the template label goes. The BPS 405 error template has this template label: 405 HEAD Request:

    RewriteCond %{REQUEST_METHOD} ^HEAD [NC]
    RewriteRule ^ – [F]

  • Profile picture of AITpro Admin

    AITpro Admin replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 6 hours ago

    All Bots make HEAD Requests first and then may use a GET Request after the HEAD Request. Sometimes HEAD Request are seen as GET Requests.

    Note: SemrushBot.* would match: SemrushBot/7~bl; +http://www.semrush.com/bot.html)

    Try this code:
    # REQUEST METHODS FILTERED
    # If you want to allow HEAD Requests use BPS Custom Code and copy
    # this…[Read more]

  • Profile picture of Ariel

    Ariel replied to the topic Crawlers (semrush, gtpbot, meta) block in the forum BulletProof Security Pro 4 days, 6 hours ago

    Thanks for your prompt reply. Actually, I was using the uptimerobot tweak already, so this is what I have now in place:

    # REQUEST METHODS FILTERED
    RewriteEngine On
    # 1. ALLOW UptimeRobot and approved crawlers to make HEAD requests
    # If HEAD *AND* UA matches an allowed bot, stop processing here and let the request through normally.
    RewriteCond…[Read more]

  • Load More

Login

Create a Login Account

Search Forums





Topic Tags

3rd Party Apps 403 error 404 error 500 error @font-face admin-ajax.php Apache Module apostrophe AutoRestore BackupBuddy BackUpWordPress bbPress better wp security BlueHost Bonus Custom Code BPS Pro Upgrade BuddyPress BulletProof Security Pro Upgrade Cache CAPTCHA CloudFlare Contact Form 7 cpanel Cron CSS cURL Custom Code Custom php.ini Setup DB Backup DSO DSO Setup Steps error_log F-Lock failed to open stream FastCGI fatal error flush_rewrite_rules GWIOD HEAD Request htaccess Redirect Code HTTP Error Log Idle Session Logout ini_set Options installation iPage Jetpack JTC Anti-Hacker JTC Anti-Spam login Login Security mailchimp maintenance mode Malware Scanner ManageWP ModSecurity mod_fcgid mod_security MScan multisite network NextGen NextGen Gallery open_basedir parenthesis php.ini handler php error PHP Error Log php errors php handler Plugin Firewall post.php Pre-installation Wizard Quarantine Redirect S-Monitor search Security Log Security Log Entries Setup Wizard single quote single quote code character Sucuri timthumb tools.php UAEG uploads Uploads Anti-Exploit Guard VaultPress W3 Total Cache W3TC wget Whitelist Rules WooCommerce Wordfence wordpress wp-admin wp-config.php WP Super Cache XAMPP xmlrpc

Proudly powered by WordPress, BuddyPress and bbPress.

Skip to toolbar
  • About WordPress
    • WordPress.org
    • Documentation
    • Learn WordPress
    • Support
    • Feedback
  • Log In
  • Register
  • Activate
  • Video Tutorials
  • BPS Pro Forum
    • Read Me First
    • BPS Pro Troubleshooting Steps
    • Post New Forum Topic
  • BPS Free Forum
    • Read Me First
    • BPS Free Troubleshooting Steps
    • Post New Forum Topic
  • Activity
  • Buy BPS Pro
  • BPS Pro Features
  • BPS Pro Download
  • Testimonials
  • Contact
  • Menu
    • Log in
    • Register
    • Activate
    • BPS Pro Forum
    • BPS Free Forum
    • Activity
    • Video Tutorials
    • Buy BPS Pro
    • BPS Pro Download
    • BPS Pro Features
    • Testimonials
    • Contact