Prepare Apache Access log for analysis

When you get your access logs it can be in various forms. Before you start them process you have to split one file to only some part or on the contrary put together few files.

Join more files to one

If you have two or more files, you can put them together with these unix commands:

rm output_log.txt
for i in *.log; do cat $i >> output_log.txt; done

All files in actual directory:

• day060101.log
• day060102.log
• day060103.log
• day060104.log

are now connected in one file named output_log.txt.

Get only part from one file

If you get for example access log from one or more months, but you want to explore log only from one week or one day. There is easy way how to do it:

From one file month0601.log we want only records for Jan 1st 2006. There are lines like this one:

domain.com - - [01/Jan/2007:02:42:12 +0200] "GET /index.html
HTTP/1.1" 200 170 "http://www.google.com/"
"Mozilla/5.0 (Windows)"

And we get only lines with string: [01/Jan/2006:

grep "\[01/Jan/2006" month0601.log > output_log.txt

To get lines from one week (Jan 1 – Jan 7) use this:

grep "\[0[1-7]/Jan/2006" month0601.log > output_log.txt