Scoop.it 403 Error – Cannot connect to this url returned a 403

[AITpro Admin]

Email Question:

Trying to use scoop.it for XML-RPC remote posting, but BPS triggers 403 error when I try to connect my sit in scoop.it dashboard. How can I prevent the problem, so I can use scoop.it? I’ve googled my brains out on this without success and just now finally had the sense to turn BPS off to see if it was the issue. Sure ’nuff. But as a follow the cookbook style web guy I dont’ know how to fix the problem. Thoughts?

[AITpro Admin]

Check your BPS Security Log for log entries relating to Scoop.it and post one of them here.

[Tom Harnish]

BPS SECURITY / HTTP ERROR LOG============================================================   That’s it. No entry. BPS was off so we could use scoop.it, turned BPS on, checked status and BPS said A-OK, tried to connect scoop.it but got 403 error. Checked log and found nothing. Refreshed page, still nothing. Turned BPS back off (default .htaccess.)–busy press day and need to use scoop.it.

[AITpro Admin]

I have looked up what Scoop.it is since I have never heard of it before.  I have a basic understanding of how it works and what it does.  I see that you can add some iframe code to your sidebar, which acts like a widget of sorts.

What I need to know from you is what exactly are you doing, how are you using Scoop.it, any URL’s related to the problem, all other specific exact details about how you are using Scoop.it and what is not working exactly.  Thanks.

[AITpro Admin]

I created a free Scoop.it account, but I am having a seriously hard time figuring this out.  The help information is very vague and fluff oriented.  I am in the Scoop.it Dashboard so how do I connect to my site or even add a URL to my site???

[Tom Harnish]

We use it to easily annotate and post web finds to our blog at teleworkresearchnetwork.com
 Next to your account name, top right, use pull down menu and select Settings, then Sharing Options, Scroll down to WordPress and enter URL, user name and password. Ta-dah (or not).

[AITpro Admin]

Yep I finally figured that out and was able to create a Scoop from one of my sites.  No errors it worked perfectly fine.  Connects to my site when I click on the Image file in the Scoop.  So the only thing I can think of is the problem might be that you are using unsafe/dangerous coding characters in your URL or Title.  Like the apostrophe for example, which is THE MOST dangerous coding character that there is – the single quote coding character.

Post the actual URL or something that I can look at.  So far I have absolutely nothing to look at and have no idea at all what is going on.  It works for me.

[Tom Harnish]

Sure appreciate your efforts, especially with so little to go on. I’m stumped too. URL or title of what?  I got the 403 error just trying to connect WordPress in Scoop settings.

[AITpro Admin]

It works perfectly for me and I even tested adding an apostrophe and it still works fine.  I cannot find any problems it works perfectly fine for me.  Send me a screenshot of the error and exactly when where and how it happens.  So far you have given me nothing to work with.

[AITpro Admin]

In your contact email you said this was for BPS Pro.  I checked the link above and you have BPS free installed on that site since I can access the /bulletproof-security/readme.txt file (shown below from that site) – that would not be possible with BPS Pro because the Plugin Firewall does not allow external access to the plugins folder.  I will move this Topic to the BPS free Forum. Thanks.

=== BulletProof Security ===
Contributors: AITpro
Donate link: http://www.ait-pro.com/aitpro-blog/331/bulletproof-security-plugin-support/bulletproof-security-donations-page/
Tags: bulletproof, security, secure, htaccess, chmod, maintenance, plugin, private, privacy, protection, permissions, 503, base64, injection, code, encode, script, attack, hack, hackers, block, blocked, prevent, prevention, RFI, XSS, CRLF, CSRF, SQL Injection, vulnerability, website security, WordPress security, security log, logging, HTTP log, error log
Requires at least: 3.0 
Tested up to: 3.5.1 
Stable tag: .48 

[Tom Harnish]

I purchased the Pro version and have it installed on another site, but removed it from this one (http://undress4success.com) because of a different problems I was having with it. I will reinstall it here when we have this sorted out.

[AITpro Admin]

UPDATE:  How to add this modification to BPS Custom Code to save it permanently.
1. Copy the modified code below (the java user agent has been removed) to this BPS Root Custom Code text box: CUSTOM CODE BPSQSE BPS QUERY STRING EXPLOITS
2. Click the Save Root Custom Code button.
3. Go to the BPS Security Modes page and click the Root Folder BulletProof Mode Activate button.

# BEGIN BPSQSE BPS QUERY STRING EXPLOITS
# The libwww-perl User Agent is forbidden - Many bad bots use libwww-perl modules, but some good bots use it too.
# Good sites such as W3C use it for their W3C-LinkChecker. 
# Use BPS Custom Code to add or remove user agents temporarily or permanently from the 
# User Agent filters directly below or to modify/edit/change any of the other security code rules below.
RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|winhttp|clshttp|loader) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]
RewriteCond %{THE_REQUEST} (\?|\*|%2a)+(%20+|\\s+|%20+\\s+|\\s+%20+|\\s+%20+\\s+)(http|https)(:/|/) [NC,OR]
RewriteCond %{THE_REQUEST} etc/passwd [NC,OR]
RewriteCond %{THE_REQUEST} cgi-bin [NC,OR]
RewriteCond %{THE_REQUEST} (%0A|%0D|\\r|\\n) [NC,OR]
RewriteCond %{REQUEST_URI} owssvr\.dll [NC,OR]
RewriteCond %{HTTP_REFERER} (%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{HTTP_REFERER} \.opendirviewer\. [NC,OR]
RewriteCond %{HTTP_REFERER} users\.skynet\.be.* [NC,OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=(http|https):// [NC,OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=(\.\.//?)+ [NC,OR]
RewriteCond %{QUERY_STRING} [a-zA-Z0-9_]=/([a-z0-9_.]//?)+ [NC,OR]
RewriteCond %{QUERY_STRING} \=PHP[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} [NC,OR]
RewriteCond %{QUERY_STRING} (\.\./|%2e%2e%2f|%2e%2e/|\.\.%2f|%2e\.%2f|%2e\./|\.%2e%2f|\.%2e/) [NC,OR]
RewriteCond %{QUERY_STRING} ftp\: [NC,OR]
RewriteCond %{QUERY_STRING} (http|https)\: [NC,OR] 
RewriteCond %{QUERY_STRING} \=\|w\| [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)/self/(.*)$ [NC,OR]
RewriteCond %{QUERY_STRING} ^(.*)cPath=(http|https)://(.*)$ [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*script.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*embed.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^e]*e)+mbed.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*object.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^o]*o)+bject.*(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (\<|%3C).*iframe.*(\>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^i]*i)+frame.*(>|%3E) [NC,OR] 
RewriteCond %{QUERY_STRING} base64_encode.*\(.*\) [NC,OR]
RewriteCond %{QUERY_STRING} base64_(en|de)code[^(]*\([^)]*\) [NC,OR]
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2}) [OR]
RewriteCond %{QUERY_STRING} ^.*(\(|\)|<|>|%3c|%3e).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(\x00|\x04|\x08|\x0d|\x1b|\x20|\x3c|\x3e|\x7f).* [NC,OR]
RewriteCond %{QUERY_STRING} (NULL|OUTFILE|LOAD_FILE) [OR]
RewriteCond %{QUERY_STRING} (\.{1,}/)+(motd|etc|bin) [NC,OR]
RewriteCond %{QUERY_STRING} (localhost|loopback|127\.0\.0\.1) [NC,OR]
RewriteCond %{QUERY_STRING} (<|>|'|%0A|%0D|%27|%3C|%3E|%00) [NC,OR]
RewriteCond %{QUERY_STRING} concat[^\(]*\( [NC,OR]
RewriteCond %{QUERY_STRING} union([^s]*s)+elect [NC,OR]
RewriteCond %{QUERY_STRING} union([^a]*a)+ll([^s]*s)+elect [NC,OR]
RewriteCond %{QUERY_STRING} \-[sdcr].*(allow_url_include|allow_url_fopen|safe_mode|disable_functions|auto_prepend_file) [NC,OR]
RewriteCond %{QUERY_STRING} (;|<|>|'|"|\)|%0A|%0D|%22|%27|%3C|%3E|%00).*(/\*|union|select|insert|drop|delete|update|cast|create|char|convert|alter|declare|order|script|set|md5|benchmark|encode) [NC,OR]
RewriteCond %{QUERY_STRING} (sp_executesql) [NC]
RewriteRule ^(.*)$ - [F]
# END BPSQSE BPS QUERY STRING EXPLOITS

Older General Additional Info (just for reference – does not apply to the newer solution above):
_________________________________
After going around in circles (the help on the Scoop.it site needs some work) I finally figured out where to go. Thanks for the screenshot you sent me. That made finding this needle in this haystack much simpler. Scoop.it sends the request with java in the User Agent. So you need to remove/delete java| from this security filter in your root .htaccess file below.

RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|java|winhttp|clshttp|loader) [NC,OR]

Security Filter With java removed/deleted from the security filter

RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|winhttp|clshttp|loader) [NC,OR]

[Tom Harnish]

That’s it! The issue was that their use of Java, which I didn’t understand. Good job! Really appreciate your concerted effort to figure this out.

[Scoop.it Support]

[Topic has been merged into this relevant Topic]

By investigating a problem one of our clients has in connecting Scoop.it to their WordPress site, it was found that their BPS plugin blocks XML-RPC requests that Scoop.it uses to post to WordPress: by disabling the BPS plugin and deleting the htaccess file, the connexion worked fine and Scoop.it originating content was posted through XML-RPC to the WordPress site. Below is also the log file of that client showing the blocked XML-RPC requests.

Can you please provide us with a whitelist rule or instructions so that our client can enjoy the benefits of their Scoop.it license while using BPS?

For more information on Scoop.it’s WordPress integration:
– http://enterprise.scoop.it/scoop-it-for-wordpress/
– http://feedback.scoop.it/knowledgebase/articles/32038-how-can-i-integrate-scoop-it-to-my-wordpress-blog

Thanks,

The Scoop.it support team
[Security Log file deleted as it did not contain any relevant log entries for scoop.it and only contained blocked hacking attempt log entries]

[AITpro Admin]

@ Scoop.it Support – Is the User Agent java fix still relevant in this forum topic or are you no longer using java in your User Agent string?

[Author]

Posts