mod_security meta-character anomaly detection alert – repetitive non-word characters

[Bob]

Greetings,

Over the weekend my site freaked out on some apparently bad code from iTunes (an attempt to add a product image and link to a widget on Friday, 17Jan) which triggered my web host’s server to suspect a brute force attack (curiously, two days later on Sunday 19Jan), which then blocked my IP address.

After my host removed the block my site (and sandbox of same, but not any other sites on my shared account even tho also running BPS) started to experience/exhibit a variety of issues, in particular affecting (or stemming from) BPS-Free to the point that any attempt to add CUSTOM CODE or even generate a new secure.htaccess instead produces (and continues now) ‘Forbidden – 403’ messages.

Here’s the info that my host provided/see in their logs and their recommendation (for BPS developers):

“Hello Bob,

It appears too many server security rules were triggered from your IP address. What occurs after too many rules are flagged the server goes into brute force defense mode and blocks your IP at the firewall. This only affects your IP address, others are able to view the website fine during this time.  Security rule triggered by 50.124.224.216 (US/United States/50-124-224-216.drr06.mskg.mi.frontiernet.net): 5 in the last 300 secs – Sun Jan 19 18:35:49 2014  I have removed your IP address from the server’s firewall and you should be able to reach your website fine now.

The security rule that was triggered was:

The security rule that was triggered was:
[Fri Jan 17 18:24:41 2014] [error] [client 50.124.224.216]
Pattern match "\\\\W{4,}" at ARGS:content.
[msg "Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters"]
[data "Matched Data: \\x22>"]
[ver "OWASP_CRS/2.2.9"] [maturity "9"] 
[accuracy "8"] 
[hostname "smoothgrind.net"] 
[uri "/wp-admin/post.php"] 
[unique_id "Utnl6a3hFSAABLqtpm0AAAAa"]

You may want to correct or replace your coding so it does not trigger the above code. This rule is on the server for security reasons as the method is often used by hackers. If necessary we can place an exception rule on the account but is not recommended as it will no longer be protected by these types of attacks.”

– and –

“As mentioned, those rewrites simply need to be updated to the newer Apache format (Apache 2.2 with RUID2). The developer of the script you are trying to use should have information on that.”

-> Long story short, my site nonetheless continues to experience various issues, and more 403’s, even after doing a LKG (last-known-good) restore of my sandbox site Sunday night.  Spent Monday and yesterday trying to troubleshoot, finally deciding last night I would just uninstall BPS from my sandbox site and reinstall clean (i.e., removing all .htaccess/backups prior to reinstall)

… but when I went to Plugins > New > Search BPS > Install the process never got started, instead the site stalled and then the server again blocked my IP.

This morning, after being unblocked again, I attempted the install once more … and it succeeded and activated, but trying to go to Settings just now it stalled, probably going to block me once more (hopefully I can post this before it does!). Far as I can tell, I think my host is correct that something is amiss with the BPS code for the newer Apache version.  Do you agree? Got a fix or workaround? Else, any other ideas/recommendations?  I’m using latest Chrome and Firefox, WP 3.8, BPS .49.8.

Appreciate all assistance!
Bob

[AITpro Admin]

I don’t think these problems are directly related to BPS, but BPS could be a factor in the equation.

mod_security is installed and configured on your Server with this SecRule below, which is what is being triggered to produce the error above that you posted.

SecRule ARGS "\W{4,}" "phase:2,capture,t:none,t:urlDecodeUni,block,id:'960024',rev:'2',ver:'OWASP_CRS/2.2.9',maturity:'9',accuracy:'8',msg:'Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters',logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',setvar:tx.anomaly_score=+%{tx.warning_anomaly_score},setvar:'tx.msg=%{rule.msg}',setvar:tx.%{rule.id}-OWASP_CRS/WEB_ATTACK/COMMAND_INJECTION-%{matched_var_name}=%{tx.0}"

So what you need to determine now is what is triggering this mod_security SecRule to generate this error.  From your description of what you have tried so far it sounds like the issue is not directly related to BPS and it is hard to tell at this point where the issue/problem really is so do these troubleshooting steps below to isolate where the problem actually is coming from (theme, plugin, etc).

Deactivate all of your WordPress plugins except for BPS then do the BPS troubleshooting steps in the link below.  Let me know the results at this point.
http://forum.ait-pro.com/forums/topic/read-me-first-free/#bps-free-general-troubleshooting

Also if your Host Server is configured in a way that allows you to turn Off mod_security on your website then you can do that by using the .htaccess code in the link below.
http://forum.ait-pro.com/forums/topic/how-to-turn-off-mod-security-mod_security-secfilterengine-off/

[Bob]

Okay, here’s what I’ve done so far … most all the first t/s steps except for re-activating each plugin one-at-a-time: do the error log entries re: admin/options.php and/or 404’s (and improvement over the 403’s) attempting to generate a new .htaccess indicate something before I start the plugin tests?

1. Error Log has mentioned BPS many times, e.g.:

HTTP_REFERER: http://smoothgrind.net/wp-admin/admin.php?page=bulletproof-security/admin/options.php
REQUEST_URI: /wp-admin/admin.php?page=bulletproof-security/admin/options.php

2. Plugin compatibility review: nothing (NLIC, Resolved or not listed)

3. .htaccess Backup successful

4. Attempt to create htaccess produces 404 (likewise for attempt to activate root folder bulletproof mode)
5. Deactivate wp-admin htaccess File = success: “The wp-admin htaccess file has been Deactivated.”

6. No Login Security & Monitoring tab: nothing to turn off (or on)
– Also, not using Brute Force Login Protection bonus code); user logins required for comments & bbPress forums

7. One-by-one plugin re-activation: (pending)

Not aware if Host Server will allow for the mod_security off instruction … think (if necessary per last reply) I will first work thru the plugin reactivation tests just to confirm that possibility(-ies).  Thanks.

Thanks for the help!

[AITpro Admin]

I just thought of something that could be triggering mod_security to see something malicious – false alert/false flag.  When you go to the BPS Settings page/htaccess Core/options.php page the Source Code of the page will output both the Root and wp-admin .htaccess code.  This is not visible to you except for if you go to the htaccess File Editor tab page and then you will see the outputted Source Code displayed in the .htaccess file’s editing windows.

Based on what mod_security is detecting as malicious Source Code I believe this BPS .htaccess security rule is being falsely interpreted as malicious code by the SecRule.

RewriteCond %{QUERY_STRING} ^.*(\x00|\x04|\x08|\x0d|\x1b|\x20|\x3c|\x3e|\x7f).* [NC,OR]

This security rule may also be falsely detected by the mod_security SecRule as being malicious code

RewriteCond %{QUERY_STRING} (\.{1,}/)+(motd|etc|bin) [NC,OR]

So what you need to do to confirm this is use FTP or your web host control panel file manager and manually edit both the Root and wp-admin htaccess files and remove the first security rule from those files and test things.  If the issue is still occurring then remove the second security rule from those files and test things.

[AITpro Admin]

If you look at where mod_security if seeing this malicious code it is logging the source URI as:  /wp-admin/post.php, which could mean that some plugin (or your Theme) that you are using that does something on the WordPress Post page is triggering this false alert or it could mean that the BPS wp-admin .htaccess file security rules above could be triggering this false alert.  The BPS wp-admin .htaccess file applies its security rules within the /wp-admin/ folder throughout the WordPress Admin area, but that is at the Server configuration level so that would mean it would not be an outputted Source Code issue and would be the .htaccess code/rules used in the wp-admin .htaccess file itself that is causing mod_security to generate a false alert/false flag.  Let me know what happens after removing the security filter above in my previous post.

Pattern match "\\\\W{4,}" at ARGS:content.
[msg "Meta-Character Anomaly Detection Alert - Repetative Non-Word Characters"]
[data "Matched Data: \\x22></ found within ARGS:content: The Mind Master\\xe2\\x80\\x99s Map - Naes Publishing\\x22 target=\\x22_blank\\x22>"]
[uri "/wp-admin/post.php"]

[AITpro Admin]

Found the problem:  it is the mod_security SecRule itself.

This pattern match \\\\W{4,} is \W{4,} without the extra Regex backslashes used to escape the single backslash used to denote “not a word character”, which translates into:  Match any character that is not a word character (alphanumeric & underscore) 4 or more times.  So the pattern is looking for characters that are not A-Z a-z 0-9 or an underscore that repeat 4 or more times in a String, which is insane….

That means the majority of the Root and wp-admin .htaccess file code matches that pattern and pretty much everything else under the sun matches that pattern – insane.

To see what I mean go to this website:  http://www.regexr.com/

Copy and paste the entire BPS Query String Exploits code into the window and add this Regex in the top window \W{4,}

Summary:  That SecRule is a configuration mistake and is too generalized/restrictive.  It either needs to be fixed or deleted as it will match just about anything and will break just about everything. The problem is the mod_security SecRule itself.

[Bob]

Okay, had to use the default secure.htaccess and wpadmin-secure.htaccess from File Editor as previous backups had only slightly more instructions than the default.htaccess produced via first t/s steps (i.e., not much if anything being blocked).

Of those two new files, secure. had both of the strings you mentioned, and wpadmin-secure. only had the second string; anyway, commented out the first string in secure. and amde no change to new wpadmin-secure. > attempt to generate a new secure. produced a 403.  Likewise, after commenting the second string in both files, another 403.

Given all the other plugins are deactivated, that leaves the  Theme (Leaf 1.1.2, with a child for the look I want) as the possible trigger you mention.  Not sure how to read into those files to tell if they’re the culprit.

add: Not sure if this will help … my Host is piping the server firewall log files to my account … here’s what’s been produced so far today (before the comment steps just completed):

--fb69a223-A--
[22/Jan/2014:05:52:02 --0800] Ut-NAK3hFSAADBcTb60AAAAV 64.246.165.160 42212 173.225.21.32 80
--fb69a223-B--
GET /comments/feed/ HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Pragma: no-cache
Accept: */*
Host: smoothgrind.net
Referer:
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; .NET CLR 1.1.4322; MEGAUPLOAD 1.0)
0: en-us,fr-be;q=0.5, Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts; .NET CLR 1.1.4322; MEGAUPLOAD 1.0)

--fb69a223-F--
HTTP/1.1 400 Bad Behavior
X-Powered-By: PHP/5.3.28
Status: 400 Bad Behavior
Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
Expires: Fri, 24 Jan 2014 13:52:00 GMT
Vary: Accept-Encoding,User-Agent
Pragma: no-cache
Content-Length: 908
Connection: close
Content-Type: text/html

--fb69a223-H--
Stopwatch: 1390398720667086 2297041 (- - -)
Stopwatch2: 1390398720667086 2297041; combined=310, p1=31, p2=269, p3=0, p4=0, p5=10, sr=0, sw=0, l=0, gc=0
Producer: ModSecurity for Apache/2.7.7 (http://www.modsecurity.org/).
Server: Apache
Engine-Mode: "ENABLED"

--fb69a223-Z--

[AITpro Admin]

See my previous post. http://forum.ait-pro.com/forums/topic/apache-version-update-required-getting-403-errors/#post-12581 You were probably posting about the same time I posted my post.  The mod_security SecRule is bunk and either needs to be fixed or deleted.

[Bob]

Alright, give me a sec to digest all that! 😉

[AITpro Admin]

Basically what I am saying is that the SecRule could be triggered by anything under your site since that pattern match would match pretty much any code in WordPress, Plugins or Themes.  There is no way to isolate what is triggering it because pretty much everything would trigger that SecRule.  It could be BPS or it could be BPS and all of your Plugins and your Theme or WordPress itself, all plugins and your Theme – everything under the sun.  Does that make the issue/problem clearer?

[Bob]

Hmm, well, I don’t get what RegExr is supposed to do after I pasted the code in the first (large) window and the \W{4,} in the top …
Anyways, moving on … which section of Custom Code would I add the ‘How to turn off mod_security’?
As a longer term/better solution, is this something best to alert my Host to dealing with on the server side?

EDIT: My bad … eyes are weary reading code!

[AITpro Admin]

What happens is in the main window the code that matches the Regex pattern that you enter in the top window gets highlighted in blue to show which code matches the pattern.  Pretty much every single BPS security rule matches that pattern.

How to turn mod_security Off in an .htaccess file/code
http://forum.ait-pro.com/forums/topic/how-to-turn-off-mod-security-mod_security-secfilterengine-off/

You won’t need to tell your Host anything.  That SecRule will cause so many problems for your Host that they will eventually remove it.  😉

 

[Bob]

Rats.  Manual update with the ‘turn off’ and still 403’s … back to the Host I go!  If they provide anything of related value I’ll update here for future reference.
I’ll give ‘em a shove all the same 😉

Thanks for your expertise!!

[Bob]

Morning,

Well, Host provided another error message in their logs Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., which I searched and found this BPS post: http://forum.ait-pro.com/forums/topic/request-exceeded-the-limit-of-10-internal-redirects/

… so I tried the suggestions from that article, and some other things: all (still) fail.  Here’s my notes of everything I tried today (I find my 3rd test most curious – no .htaccess at all and still 403 on both sites?):

24Jan –

Note: live site tests using Firefox; sandbox using Chrome

Note: live site has all plugins deactivated; sandbox all active (and same as live)

1. Per http://forum.ait-pro.com/forums/topic/request-exceeded-the-limit-of-10-internal-redirects/#post-1418:

1a. “the error is gone when I switch to default wordpress htaccess.” – Not true for me, either live or sandbox.
1b. Added # .htaccess Fix for Infinite Loops:  403 both sites
1c.  commented out all Query String Exploits: still 403 both sites

3. live & sandbox – With no (none, zip, zilch) .htaccess in root or wp-admin: still 403s.

4. sandbox – can create and activate maintenance.htaccess; test activating root folder bulletproof mode: 403
4a. live – also can create & activate maintenance, but this time attempt to create new secure.htaccess: 404 – Not Found (not a 403)

5. using a fresh secure.htaccess (copied from Editor tab), no custom code, com’d lines 34 (# DirectoryIndex index.php index.html /index.php) plus all ErrorDocument lines, and line 64 RedirectMatch 403 => produces 404 at http://smoothgrind.net/[removed link to wp-admin area] (/wp-admin folder does not have any .htaccess)

I’m still following you that you think the mod_security is at the heart of this, yet Host reports that (Rule 960024 on their records) was removed a few days ago … like your prediction was right and they already remedied it.

Any other ideas? (scratch the ‘move to deserted island’ option … wife won’t let me go! 😉

Thanks,
Bob

[AITpro Admin]

I’m still following you that you think the mod_security is at the heart of this, yet Host reports that (Rule 960024 on their records) was removed a few days ago … like your prediction was right and they already remedied it.

Yep I imagine that your Host quickly figured out that that mod_security SecRule would increase their support time workload exponentially since that rule will be triggered by just about anything.  ha ha ha.  🙂

1a. “the error is gone when I switch to default wordpress htaccess.”  Not true for me, either live or sandbox.

Are you saying that when you use BPS Default Mode / a standard WordPress .htaccess code/file the infinite redirect loop error still occurs?

5. using a fresh secure.htaccess (copied from Editor tab), no custom code, com’d lines 34 (# DirectoryIndex index.php index.html /index.php) plus all ErrorDocument lines, and line 64 RedirectMatch 403 => produces 404…

Yes, that will happen on some Hosts.  If you are using Root folder BulletProof Mode then on some Hosts you MUST also use wp-admin BulletProof Mode when Root folder BulletProof Mode is activated.  If you have deactivated Root Folder BulletProof Mode then wp-admin BulletProof Mode is NOT required on these particular Host Server configurations.

4. sandbox – can create and activate maintenance.htaccess; test activating root bulletproof mode: 403
4a. live – also can create & activate maintenance, but this time attempt to create new secure.htaccess: 404 – Not Found (not a 403)

Maintenance Mode is completely redesigned in BPS .49.9 and no longer uses or has a maintenance .htaccess file so don’t bother with testing that as it will be removed.

http://forum.ait-pro.com/forums/topic/maintenance-mode-guide-read-me-first/

[Author]

Posts