Security Log – What matters and what does not matter

Home Forums BulletProof Security Pro Security Log – What matters and what does not matter

Viewing 15 posts - 1 through 15 (of 17 total)
  • Author
    Posts
  • #2351
    Bro Ignatius Mary
    Participant

    Dozens and dozens of Security Log 403 errors are being logged. I do not understand this. I give some examples below, where the REQUEST_URL is valid. Why the error?

    >>>>>>>>>>> 403 GET or Other Request Error Logged - February 28, 2013 - 4:43 pm <<<<<<<<<<<   
    REMOTE_ADDR: 184.169.203.101 
    Host Name: ec2-184-169-203-101.us-west-1.compute.amazonaws.com 
    HTTP_CLIENT_IP: 
    HTTP_FORWARDED: 
    HTTP_X_FORWARDED_FOR: 10.196.14.144, 10.164.21.113 
    HTTP_X_CLUSTER_CLIENT_IP: 
    REQUEST_METHOD: GET 
    HTTP_REFERER: 
    REQUEST_URI: /bubba/900 
    QUERY_STRING: 
    HTTP_USER_AGENT: UnwindFetchor/1.0 (+http://www.gnip.com/)

    And what do these mean? Is someone trying to break in to js files?

    >>>>>>>>>>> 403 GET or Other Request Error Logged - February 28, 2013 - 4:26 pm <<<<<<<<<<< 
    REMOTE_ADDR: 64.125.188.25 
    Host Name: n1.dfw.ces.cvnt.net 
    HTTP_CLIENT_IP: 
    HTTP_FORWARDED: 
    HTTP_X_FORWARDED_FOR: 
    HTTP_X_CLUSTER_CLIENT_IP: 
    REQUEST_METHOD: GET 
    HTTP_REFERER: 
    REQUEST_URI: /bubba/wp-content/plugins/bulletproof-security/admin/js/bulletproof-security-admin-2.js?ver=3.5.1 
    QUERY_STRING: 
    HTTP_USER_AGENT: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
    #2357
    AITpro Admin
    Keymaster

    The first log entry is the UnwindFetchor bot making a HEAD Request to your website.  If you would like to allow bots to make a HEAD Request then remove/delete HEAD from the Request Method Filter as shown in this Forum Topic:  http://forum.ait-pro.com/forums/topic/mailchimp-tracking-code-causing-403/#post-1640  .  Or if you choose not to allow this then you will see millions of these until the end of time.  You can just disregard them.

    The second log entry looks like some sort of recon for the bulletproof security js file.  This is Forbidden by the Plugin Firewall.  The Plugin Firewall forbids any external access to any plugin files, including the BPS Pro plugin files.

    The only log entries that need any attention from you would be these in the Security Log Read Me First Forum Topic link below:

    http://forum.ait-pro.com/forums/topic/security-log-http-error-log-read-me-first/

    This Forum Topic link below explains that most log entries can be ignored and why:

    http://forum.ait-pro.com/forums/topic/security-log-security-log-403-errors/#post-1694

     

    #3086

    Uhm… If the log events in question will occur in the “millions… until the end of time” and if they’re so unimportant that “you can just disregard them” and if they are not among the “log entries that need any attention”, then why on earth does the plugin bloat the log with them at all?

    Is it not obvious that this will require manual copying and deletion of the rapidly growing log every day or two?

    The plugin effectively imposes the following choice:

    Either (a) remove the filter, even though you might want it, or (b) keep the filter, but dedicate yourself to a life of manual log file management, since the log will grow like crazy when the filter’s on and will prompt you every day or two with a warning banner on the dashboard, or (c) turn off logging.

    Please note that neither (a) nor (b) nor (c) is satisfactory for users who want to retain the filter, retain logging, and have a life not given over to managing a log file filled with unhelpful content.

    Wouldn’t it make more sense to write trivial events (if at all) to a secondary log file (rolling, with only the most recent few files retained), to write only the most important events to the main log, and then to have a size monitor with dashboard warnings only for the important log?

    Frequent warning banners and calls for manual file management because the huge log is yet again filled with trivial info— let’s just say that this feature seems ill conceived.

    #3092
    AITpro Admin
    Keymaster

    For BPS Pro folks all log files are automatically zipped and emailed based on user preferences/options settings in S-Monitor. There is no manual maintenance required for any BPS Pro log files since the entire system is automated.

    For BPS Free folks the next version release of BPS will contain an additional option setting to exclude frequent unimportant/trivial logged events based on the User Agent String.  S-Monitor cannot be added to BPS free due to the complexity of how S-Monitor is tied into BPS Pro – it is the central monitoring and alerting system for BPS Pro.  Trying to include this feature in BPS Free would increase our support times to an unsatisfactory level and have a negative impact on BPS Pro Development and Growth.

    #3095

    For BPS Free folks the next version release of BPS will contain an additional option setting to exclude frequent unimportant/trivial logged events based on User Agent.

    That should help. Thanks.

    Since Free should induce or persuade users to buy Pro, it seems important for Free to avoid annoyances that make users doubt the design judgment behind Pro.

    #3097
    AITpro Admin
    Keymaster

    The Security/HTTP Error Log should really only be used for troubleshooting possible plugin or coding conflicts/issues/problems and then be turned off.  There is no point in logging all the typical scanning/probing/reconning/sniffing/scraping/hacking that goes on day in and day out.

    #3114

    Agreed. So is it on or off by default?

    #3119
    AITpro Admin
    Keymaster

    I recommend turning it Off.

    #3123

    So do I. But is it on xor off when you first install BPS Free? What is the default setting for logging?

    #3125
    AITpro Admin
    Keymaster

    Oh of course it is set to On for a number of technical reasons and Support reasons.

    QASI0378:  BPS Feature Trial Testing Results and Support Impact:

    Setting Error Logging to Off by default

    Support Impact:  Very Bad.

    Severe user confusion resulting in unnecessary support downtime answering questions of this general nature.

    My error log is not logging errors, is not working,  is broken, etc.

    Setting Error Logging to On by default

    Support Impact:  Very Good.

    User responses are based on adjusting personal settings and configurations.  It is assumed that error logging is working correctly.

    Additional Notes:  When users perform BPS upgrades any customizations to .htaccess code or option settings are not changed and are retained.

    #3133

    You say “Oh of course it is set to On for a number of technical reasons and Support reasons” and you “recommend turning it Off“?

    Consider two cases:
    (A) telling people which button to click to turn on logging
    (B) telling people why their log file keeps filling up and splatting on the dashboard, and explaining to people which 80% of the log’s contents they can safely ignore

    I’m surprised that (A) requires more support time than (B). But since you’ve measured the support impact, I guess that’s all there is to say! Except, of course, “Nevermind! I recommend turning it off!” 😉

    #3136
    AITpro Admin
    Keymaster

    I just go by testing results/stats and don’t really have a say or opinion about what the testers and assessors come up with.  😉  They do thorough testing with trial groups and make their assessments.  Me, I just do plain old paint by numbers and copy and paste.  LOL

    #3138
    AITpro Admin
    Keymaster

    I actually checked on this because I was curious about it.  Here is what I was told.  75% of the people looked at their Security Log file and it was blank.  On the testing sheet they marked off that Security logging did not work.  😉  The assessors also give suggestions/recommendations to improve options.  They recommended having a text message displaying “security logging is not turned on – click here to turn it on”.  This sounds like a winner to me so I checked further to see why this was not done and was told this – “if you ask the user to check his/her security log for errors and the error log is turned off then how are you going to troubleshoot the problem”.  Yeah I guess they are on the ball.  😉

    #3139

    Thanks for the follow-up. Honestly, it sounds as if the way to strike the balance is to have the log on by default, but to write only truly noteworthy events to it.

    Logging information that can safely be ignored in response to events that happen very often (e.g., harmless bot automations) seems like a good way to cause the log to fill up way too quickly.

    Another option– not as effective, since it’s just a band-aid– would be to increase the log file size threshold that causes the warning banner to appear.

    #3140
    AITpro Admin
    Keymaster

    Yeah I thought of that question and asked that too.  The answer:  IP Addresses, hostnames and User Agents can all be faked.  The Google Bot was attempting to hack the AITpro websites for over a week.  😉  The last time I checked Google does not do much hacking.  LOL

    So basically what I was told is that since it is not possible to tell if any of these variables/logged events are real or not or if they are a serious threat or not then the only logical option left is to leave it up to the user to decide what they want to do about logged events based on the User Agent.  Hence the new option in the next version of BPS to choose which User Agents to ignore/not log.

    The larger the log file the slower the BPS options page will load so that is out as well.  From what I was told 500KB causes a slight page load time increase and 1MB causes a very significant page load time increase.  😉

Viewing 15 posts - 1 through 15 (of 17 total)
  • You must be logged in to reply to this topic.