4.4 Cut the Log in Half
4.4.1 Description
Sometimes log files need to be trimmed down in size. Here is one
way to cut the log file in half.
Example One: Trimming It with tail
Flavors: AT&T, BSD
Shell: sh
Syntax:
bc
tail -number
wc [-l] file
|
Warning - Before cutting log files
in half, stop any and all processes that might have the log files open.
|
Find the number of lines in the log file:
% wc -l SYSLOG
1234567
Divide this number in half:
% echo "1234567 / 2" | bc
617283
tail can deliver the last part of
the file in two ways: by counting the distance in lines either down from the
top of the file or up from the bottom of the file.
For
counting the distance in lines from the top use
% tail +617283 SYSLOG > SYSLOG.half
For counting the distance in lines from the bottom use
% tail -617283 SYSLOG > SYSLOG.half
This process can be quickly scripted to perform this function
for you. In the script, pass through the name of the file to cut in half. Then
determine what the size is, take half that total size, and use tail
to trim it down to size.
#! /bin/sh
FILE=$1
SIZE=`cat $FILE | wc -l`
HALF=`echo $SIZE/2 | bc`
echo "Total size = $SIZE Half = $HALF"
tail +${HALF} $FILE > $FILE.cut
ls -l $FILE $FILE.cut
Line 1: Define the shell to use.
Line 2: Pass in the file that is to be cut in half.
Line 3: Determine the size of the file.
Line 4: Take half the total value of the file to be cut.
Line 5: Display the original size and what half the size is.
Line 6: Cut the file in half, counting from the top down and writing
the bottom half of the file out to a new filename.
Line
7: Display a long listing of the old and new file.
xinu 1% cutit backup.ugu.INDEX.Thu
Total size = 1634885 Half = 817442
-rw-r--r-- 1 ugu user 100455424 Sep 16 23:33 backup.ugu.INDEX.Thu
-rw-r--r-- 1 ugu user 42082249 Sep 21 20:17 backup.ugu.INDEX.Thu.cut
Reason
Sometimes you get huge log files that you cannot bring into a
vi editor or do anything with. You see buffer overrun errors and often run out
of disk space. Cutting the log files in half frees up disk space and makes the
log files more manageable from an administration standpoint.
Real World Experience
At some point a process can run out of control, creating an incredible
amount of the entries in your log file. If this were to happen over the weekend
when no one was around, your disk would more than likely be full when you arrived
at work on Monday morning. Depending on how much or what data you want, you
have a couple of options.
Stop any daemons or processes that are writing to that particular
log file. Then find disk space on another partition, disk, or through an NFS
mount point. If there is only an NFS mount point available, this might depend
on the size of the file.
If you want to
get rid of the massive number of error messages that were logged in the file,
grep those out using a pattern that exists in all
the error messages:
# grep -v "ERROR dbase offline" /var/adm/messages >/var/adm/messages.new
# mv /var/adm/messages.new /var/adm/messages.
If you don't want the file at all, null
the file fast and reclaim all the lost disk space:
# cat /dev/null > /var/adm/messages
If you have some time to manipulate the file, move it to another
area:
# mv /var/adm/messages /disk2/tmp
Then cut the file in half using the commands of the script discussed.
Other Resources
Man pages:
bc, grep,
tail, wc
© Copyright Macmillan USA. All rights
reserved.
