Thursday, January 20, 2011

Apache_log analysis

My external web host generates what seems to be standard apache access_log files. Although there are numerous (including free) report tools for this, I havent found any which are fully customisable, i.e. they all just perform some standard reports which are tunable to some extend. The main report i want is to ensure site access with respect to user (they all need to login using .htaccess), IP address and files. I can get all of these individually, but not combined. For example, I would like to know whether Fred checked the HighlyConfidential folder from ip address 1.2.3.4. Dont really care how I specify all this with the exception that I dont have any UNIX/Linux access since all is Windows based (but MySQL would be fine if this helps).

  • I would suggest using cygwin which gives you a linux bash shell. With this, you can filter by IP/file etc etc. I can't suggest any web log analyser software which is free that can do this. Might be worth looking at SawMill.

    From AliGibbs
  • You can use Splunk. This web app works on Windows and can extract any fields you want from log files. Splunk 4 has a free version (you can wait 30 days of enterprise and the go free or go and activate the free version right away).

    To get the reports you want, Splunk lets you do searches (like a search engine). You can save the searches, you can graph them and you can make dashboards.

    If you are interested you can even create a Splunk app for your particular problem.

    Rob : this looks good indeed; dont think its the most straightforward interface, but (as you say) it can probably get out anything you want in any format. found user, uri_path and clientip, but not succeded in getting it into a pivot table style report yet
    chmeee : I haven't done that myself. I guess you could do a search from the command line and parse the results into CSV. Otherwise, I have found this excel addon http://code.google.com/p/splunk-excel-addon/ that may be of help (haven't tried it myself, though)
    From chmeee
  • I would also suggest cygwin, there is a very easy installer, then I would use petit to track the types of entries that you have.

    http://opensource.eyemg.com/index.php/Petit#Cygwin

    Joe : Yup. `grep | grep | grep ` would do it.
    From

0 comments:

Post a Comment