Hotlink Protection Do Not Block Google, Bing or Yahoo

[Geoff]

I am having an issue with my Hotlinking Custom Code.  I have the following Custom Code:

# BLOCK HOTLINKING TO IMAGES
SetEnvIfNoCase Referer "^(http|https)://.*\.firstgenfirebird\.org$" whitelist
SetEnvIfNoCase Referer "^(http|https)://.*\.google\.com$" whitelist
SetEnvIfNoCase Referer "^(http|https)://.*\.yahoo\.com$" whitelist
SetEnvIfNoCase Referer "^(http|https)://.*\.bing\.com$" whitelist

<FilesMatch "\.(gif|jpg|jpeg|png|bmp|pdf)$">
Order Allow,Deny
Allow from env=whitelist
# Add Your Server IP Address
Allow from 209.240.129.103
</FilesMatch>

I have a forum which started supporting inline viewing of PDF files with the latest update.  The PDF files are being blocked from loading and removing the Filesmatch PDF corrects the issue.  I am not sure why the above Google directive is not working.  Any idea so I can block hotlinking to our PDF files again?

This is from my log file:

[403 GET Request: October 9, 2019 - 3:40 pm]
BPS Pro: 14.2
WP: 5.2.3
Event Code: BFHS - Blocked/Forbidden Hacker or Spammer
Solution: N/A - Hacker/Spammer Blocked/Forbidden
REMOTE_ADDR: 66.xxx.xx.1
Host Name: google-proxy-66-xxx-xx-1.google.com
SERVER_PROTOCOL: HTTP/1.1
HTTP_CLIENT_IP:
HTTP_FORWARDED:
HTTP_X_FORWARDED_FOR:
HTTP_X_CLUSTER_CLIENT_IP:
REQUEST_METHOD: GET
HTTP_REFERER:
REQUEST_URI: /forums/xxx/16926.pdf
QUERY_STRING:
HTTP_USER_AGENT: Mozilla/5.0 (compatible; Google AppsViewer; http://drive.google.com)

Thank you in advance – Geoff

[AITpro Admin]

Geoff – The hotlink protection code is allowing/blocking by checking the referer.  The Security Log entry shows that the HTTP_REFERER is blank.  Hotlinking typically works like this:  Someone links to your image files on your website and displays your images on their website.  When they do that (hotlink your images) their website URI is the referer.  The whitelisting rules in the hotlink protection code is used in this scenario:  Google, Yahoo, Bing, etc. index your images in the images feature for the Search Engine.  ie Google Images, etc.  This is very helpful for SEO.

So it appears that what you are doing on your website is using a Google PDF Viewer to view images.  The User Agent Security Log entry shows: Google AppsViewer from http://drive.google.com, which probably means that your inline PDF images are using the Google Drive PDF Viewer App.

You cannot whitelist by User Agent since hackers and spammers frequently use “google” in their spoofed User Agent.  So you would want to whitelist by Remote Address or Remote Host.

Example:
Remote_Host
the hostname (if available) of the client making the request – google-proxy-66-249-93-195.google.com

Remote_Addr
IP address of the client making the request – 66.249.93.195

# BLOCK HOTLINKING TO IMAGES
SetEnvIfNoCase Referer "^(http|https)://.*\.firstgenfirebird\.org$" whitelist
SetEnvIfNoCase Referer "^(http|https)://.*\.google\.com$" whitelist
# whitelist by the Google Proxy host name.
SetEnvIfNoCase Remote_Host ".*\.google\.com$" whitelist
SetEnvIfNoCase Referer "^(http|https)://.*\.yahoo\.com$" whitelist
SetEnvIfNoCase Referer "^(http|https)://.*\.bing\.com$" whitelist

<FilesMatch "\.(gif|jpg|jpeg|png|bmp|pdf)$">
Order Allow,Deny
Allow from env=whitelist
# Add Your Server IP Address
Allow from 209.240.129.103
# Or you can whitelist by the Google Proxy IP CIDR Range
Allow from 66.249.93.195/24
</FilesMatch>

[Geoff]

I will give it a try sometime in the next few days and let you know the outcome.

[Michael Amon]

Hello AITPro Admin,

For those folks whose domain is like

example.co.uk

how do they add their domain into that code? I’m asking this question because this code

SetEnvIfNoCase Referer "^(http|https)://.*\.ait-pro\.com$" whitelist

doesn’t seem to work for me at all.

[AITpro Admin]

If you have a LiteSpeed server then SetEnvIf does not work on LiteSpeed servers.  You can use a RewriteRule instead > https://store.litespeedtech.com/store/index.php?rp=/knowledgebase/90/Replace-SetEnvIf-with-RewriteRule-to-block-bad-clients.html.  If you have an nginx server then apache htaccess code will not work on nginx.

SetEnvIfNoCase Referer "^(http|https)://.*\.example\.co\.uk$" whitelist

[Author]

Posts