Unable to publish posts from OpenTopic

[Stewart Marshall]

I have been using OpenTopic – app.opentopic.com – as a content curation and distribution tool.  It allows me to post news items from the web to my WP website stewart-marshall.com.  After installing BPS no further posts from Open Topic could be published  – but when BulletProof Mode was deactivated, posts were possible from OpenTopic.

How can I activate BPS whilst also allowing access by OpenTopic?

Thanks

[AITpro Admin]

Post any BPS Security Log errors related to OpenTopic. If you do not find any errors in the BPS Security Log then post any error messages that you see on the Opentopic.com site.

[Stewart Marshall]

I’m not seeing any error reports in the BPS Security Log file (Security Log Last Modified Time: September 17, but the last entry in the log was August 17).
And according to the OpenTopic.com site the post has been published alright – but when I look at my site no post has been published.
Then when I Deactivate Root Folder BulletProof Mode, I can publish posts from OpenTopic alright.

[AITpro Admin]

hmm ok then whatever OpenTopic is doing must be some sort of java based thing or a remote thing since errors are not being logged in the Security Log.

BPS Pro 11.6+ & BPS free .53.2+
You may see this code or the 11.5+/.53.1+ code in your root htaccess file.  The code does the same exact thing and is whitelisted in the same exact way.

# REQUEST METHODS FILTERED
# If you want to allow HEAD Requests use BPS Custom Code and copy
# this entire REQUEST METHODS FILTERED section of code to this BPS Custom Code
# text box: CUSTOM CODE REQUEST METHODS FILTERED.
# See the CUSTOM CODE REQUEST METHODS FILTERED help text for additional steps.
RewriteCond %{REQUEST_METHOD} ^(TRACE|DELETE|TRACK|DEBUG) [NC]
RewriteRule ^(.*)$ - [F]
#RewriteCond %{REQUEST_METHOD} ^(HEAD) [NC]
#RewriteRule ^(.*)$ /wp-content/plugins/bulletproof-security/405.php [L]

BPS Pro 11.5+ & BPS free .53.1+

# REQUEST METHODS FILTERED
# If you want to allow HEAD Requests use BPS Custom Code and copy
# this entire REQUEST METHODS FILTERED section of code to this BPS Custom Code
# text box: CUSTOM CODE REQUEST METHODS FILTERED.
# See the CUSTOM CODE REQUEST METHODS FILTERED help text for additional steps.
RewriteCond %{REQUEST_METHOD} ^(TRACE|DELETE|TRACK|DEBUG) [NC]
RewriteRule ^(.*)$ - [F]
#RewriteCond %{REQUEST_METHOD} ^(HEAD) [NC]
#RewriteRule ^(.*)$ - [R=405,L]

BPS Pro 11.4|BPS free .53 and lower versions
Try something simple first. Edit your root .htaccess file and remove HEAD| from this code/filter.

Example: (TRACE|DELETE|TRACK|DEBUG)

# REQUEST METHODS FILTERED
# If you want to allow HEAD Requests use BPS Custom Code and 
# remove/delete HEAD| from the Request Method filter.
# Example: RewriteCond %{REQUEST_METHOD} ^(TRACE|DELETE|TRACK|DEBUG) [NC]
# The TRACE, DELETE, TRACK and DEBUG Request methods should never be removed.
RewriteCond %{REQUEST_METHOD} ^(HEAD|TRACE|DELETE|TRACK|DEBUG) [NC]
RewriteRule ^(.*)$ - [F]

If the problem is still occurring then comment out all the BPS Query String code/rules by putting a pound sign # in front of each rule:

IMPORTANT!  Do not put a pound sign in front of this rule/filter:  RewriteCond %{QUERY_STRING} (sp_executesql) [NC] and leave it as is.  Do not put a pound sign in front of the RewriteRule:  RewriteRule ^(.*)$ – [F,L]

# BEGIN BPSQSE BPS QUERY STRING EXPLOITS
# The libwww-perl User Agent is forbidden - Many bad bots use libwww-perl modules, but some good bots use it too.
# Good sites such as W3C use it for their W3C-LinkChecker. 
# Use BPS Custom Code to add or remove user agents temporarily or permanently from the 
# User Agent filters directly below or to modify/edit/change any of the other security code rules below.
#RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|java|winhttp|clshttp|loader) [NC,OR]
#RewriteCond %{HTTP_USER_AGENT} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
#RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|java|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]
...
...
...
#RewriteCond %{QUERY_STRING} \-[sdcr].*(allow_url_include|allow_url_fopen|safe_mode|disable_functions|auto_prepend_file) [NC,OR]
#RewriteCond %{QUERY_STRING} (;|<|>|'|"|\)|%0A|%0D|%22|%27|%3C|%3E|%00).*(/\*|union|select|insert|drop|delete|update|cast|create|char|convert|alter|declare|order|script|set|md5|benchmark|encode) [NC,OR]
RewriteCond %{QUERY_STRING} (sp_executesql) [NC]
RewriteRule ^(.*)$ - [F]
# END BPSQSE BPS QUERY STRING EXPLOITS

[AITpro Admin]

I tried looking at their site, but the site does not contain any information about what or how OpenTopic works.  The site has some nice graphics, but no technical info whatsoever.

Let me know what happens after commenting out all the security filters.  Thanks.

[Stewart Marshall]

Thanks for the suggestions.

The first suggestion “Edit your root .htaccess file and remove HEAD| from this code/filter” didn’t solve the problem.

But the second suggestion “comment out all the BPS Query String code/rules by putting a pound sign # in front of each rule” solved the problem.  I am now able to post freely from OpenTopic to my WP website with BPS activated 🙂

What are the security implications of implementing these two suggestions?

Many thanks

Stewart

[AITpro Admin]

Commenting out all the BPS Query string code/rules was a troubleshooting step to isolate generally where the issue is occurring.  Next, since I am unable to find any technical or help information on how OpenTopic works you would need to uncomment BPS Query String code/rules 1 line at a time until you isolate which line/rule is the one blocking OpenTopic.

Taking a logical guess I would say it is going to 1 or more of these security rules:

Uncomment all 3 rules below and test.

RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|java|winhttp|clshttp|loader) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|java|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]

If the issue/problem is Not occurring then these are the next logical rules to uncomment:

RewriteCond %{THE_REQUEST} \?\ HTTP/ [NC,OR]
RewriteCond %{THE_REQUEST} \/\*\ HTTP/ [NC,OR]

If the issue/problem is Not occurring then these are the next logical rules to uncomment:

RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=http:// [OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=(\.\.//?)+ [OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=/([a-z0-9_.]//?)+ [NC,OR]

If the issue/problem is Not occurring then these are the next logical rules to uncomment:

RewriteCond %{QUERY_STRING} ftp\: [NC,OR]
RewriteCond %{QUERY_STRING} http\: [NC,OR]
RewriteCond %{QUERY_STRING} https\: [NC,OR]

[Stewart Marshall]

Your first suggestion identified the problem as being in the three condition rules:

RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|java|winhttp|clshttp|loader) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|java|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]

So after determining this, I commented just those three, and uncommented all the other condition rules.

OpenTopic was still able to post to the WP site with BPS activated.

Thanks for all your help.

[AITpro Admin]

Ok now if you want to narrow it down even farther then the most likely User Agent string that is being blocked is going to be:  java.

First uncomment this rule and OpenTopic should still work…

RewriteCond %{HTTP_USER_AGENT} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]

…next remove “java” from these 2 security filters/rules and test OpenTopic…

RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|winhttp|clshttp|loader) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]

…The next most likely User Agent strings to remove would be:  wget and then next would be:  libwww-perl.

[Stewart Marshall]

Thanks for the next steps.

Uncommenting the rule:

RewriteCond %{HTTP_USER_AGENT} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]

worked fine.

But uncommenting and then successively removing “java”, “wget” and “libwww-perl” from the other two security filters/rules did not allow OpenTopic to publish to WP on each try.

[Stewart Marshall]

Okay – I followed your lead and tried removing “python” from the two filters/rules:

RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|winhttp|clshttp|loader) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]

And OpenTopic now works with BPS activated and all the BPS Query string code/rules uncommented.

Thanks for all your help 🙂

[AITpro Admin]

Wow!  Python really?  How unusual/unique.  Thanks for confirming what worked. 😉

Working Solution:
1.  Copy the code below (python has been removed already from this code below) to this BPS Custom Code text box:  CUSTOM CODE BPSQSE BPS QUERY STRING EXPLOITS: Modify Query String Exploit code here
2.  Click the Save Root Custom Code button.
3. Go to the BPS Security Modes page and click the Root Folder BulletProof Mode Activate button.
Note: For good measure clear your Browser cache and if you are using a caching plugin clear your caching plugin cache.

# BEGIN BPSQSE BPS QUERY STRING EXPLOITS
# The libwww-perl User Agent is forbidden - Many bad bots use libwww-perl modules, but some good bots use it too.
# Good sites such as W3C use it for their W3C-LinkChecker.
# Use BPS Custom Code to add or remove user agents temporarily or permanently from the
# User Agent filters directly below or to modify/edit/change any of the other security code rules below.
RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|nikto|curl|scan|java|winhttp|clshttp|loader) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|nikto|curl|scan|java|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]
RewriteCond %{THE_REQUEST} (\?|\*|%2a)+(%20+|\\s+|%20+\\s+|\\s+%20+|\\s+%20+\\s+)(http|https)(:/|/) [NC,OR]
RewriteCond %{THE_REQUEST} etc/passwd [NC,OR]
RewriteCond %{THE_REQUEST} cgi-bin [NC,OR]
RewriteCond %{THE_REQUEST} (%0A|%0D|\\r|\\n) [NC,OR]
RewriteCond %{REQUEST_URI} owssvr\.dll [NC,OR]
RewriteCond %{HTTP_REFERER} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{HTTP_REFERER} \.opendirviewer\. [NC,OR]
RewriteCond %{HTTP_REFERER} users\.skynet\.be.* [NC,OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=(http|https):// [NC,OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=(\.\.//?)+ [NC,OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=/([a-z0-9_.]//?)+ [NC,OR]
RewriteCond %{QUERY_STRING} \=PHP[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} [NC,OR]
RewriteCond %{QUERY_STRING} (\.\./|%2e%2e%2f|%2e%2e/|\.\.%2f|%2e\.%2f|%2e\./|\.%2e%2f|\.%2e/) [NC,OR]
RewriteCond %{QUERY_STRING} ftp\: [NC,OR]
RewriteCond %{QUERY_STRING} (http|https)\: [NC,OR]
RewriteCond %{QUERY_STRING} \=\|w\| [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)/self/(.*)$ [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)cPath=(http|https)://(.*)$ [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*script.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*embed.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^e]*e)+mbed.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*object.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^o]*o)+bject.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*iframe.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^i]*i)+frame.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} base64_encode.*\(.*\) [NC,OR]
RewriteCond %{QUERY_STRING} base64_(en|de)code[^(]*\([^)]*\) [NC,OR]
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2}) [OR]
RewriteCond %{QUERY_STRING} ^.*(\(|\)|<|>|%3c|%3e).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(\x00|\x04|\x08|\x0d|\x1b|\x20|\x3c|\x3e|\x7f).* [NC,OR]
RewriteCond %{QUERY_STRING} (NULL|OUTFILE|LOAD_FILE) [OR]
RewriteCond %{QUERY_STRING} (\.{1,}/)+(motd|etc|bin) [NC,OR]
RewriteCond %{QUERY_STRING} (localhost|loopback|127\.0\.0\.1) [NC,OR]
RewriteCond %{QUERY_STRING} (<|>|'|%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{QUERY_STRING} concat[^\(]*\( [NC,OR]
RewriteCond %{QUERY_STRING} union([^s]*s)+elect [NC,OR]
RewriteCond %{QUERY_STRING} union([^a]*a)+ll([^s]*s)+elect [NC,OR]
RewriteCond %{QUERY_STRING} \-[sdcr].*(allow_url_include|allow_url_fopen|safe_mode|disable_functions|auto_prepend_file) [NC,OR]
RewriteCond %{QUERY_STRING} (;|<|>|'|"|\)|%0A|%0D|%22|%27|%3C|%3E|%00).*(/\*|union|select|insert|drop|delete|update|cast|create|char|convert|alter|declare|order|script|set|md5|benchmark|encode) [NC,OR]
RewriteCond %{QUERY_STRING} (sp_executesql) [NC]
RewriteRule ^(.*)$ - [F]
# END BPSQSE BPS QUERY STRING EXPLOITS

[Author]

Posts