Chapter 6
File Management
Newer versions of UNIX are adding more and more files to their
flavors. A basic system can have as many as 10,000 files built in to the UNIX
operating system. These files live in as many as 30 standard directories. Although
you can use only 10 directories on a regular basis, you should try to gain an
understanding of all the directories that commonly appear within UNIX.
Almost all manufacturers now bundle an extensive amount of third-party
software with their flavor of UNIX. True, it doesn't all have to be installed,
but straight from the factory, UNIX is typically fully loaded with about every
piece of software available, including demos, games, and all the third-party
products from companies that the manufacturer has agreements with. With this
in mind, it is usually best to wipe the disk clean and rebuild the system with
the various software packages that make up UNIX so that it will fit the needs
of your environment.
Managing and working with the thousands of files that make up
UNIX becomes a greater challenge with each vendor's new release of the
operating system. In this chapter I will deal with how you can manage, move,
migrate, search, and work with files in various ways.
6.1 Copy Files with Permissions and Time Stamps
6.1.1 Description
A straight copy command ( cp) duplicates
the file but leaves it with new permissions and a new ownership. However, there
are ways to skirt around this issue and have the cp
command maintain the permissions and ownership of a file.
Example One: Copy with Permissions
Flavors: AT&T, BSD
Shells: All
Syntax:
cp -[pr] files [file|dir]
cp -[pr] dirs [dir]
You can copy files while attempting to preserve the permissions,
ownership, groups, and time stamp by using the -p
argument with your cp command:
# whoami
root
# cd /home/steve
# ls -al .cshrc
-rw-rw-r-- 1 steve staff 2426 May 5 1998 .cshrc
# cp /.cshrc /tmp
# ls -al /tmp/.cshrc
-rw-rw-r-- 1 root sys 2426 Nov 3 13:22 .cshrc
# cp -p /.cshrc /tmp
# ls -al /tmp/.cshrc
-rw-rw-r-- 1 steve staff 2426 May 5 1998 .cshrc
If the -p argument is not provided,
some flavors always attempt to set the destination file or files to the owner
of the source file if it is possible by default. There are instances where even
providing the -p argument does not change the permission
of the destination file. This occurs when the account attempting to make the
copy doesn't have permission to make such a permission change.
Using the -r option recursively copies
files down the subdirectories to the destination. When the command comes across
a symbolic link it turns that link into a directory at the destination point.
The command then proceeds through the symbolic link and continues to copy the
data that the link points at. Because the symbolic link is turned into a directory
with the cp command, you now might have to keep
the new directory in synch with the directory that the symbolic link was pointing
to, depending on your needs.
Suppose there were three directories called
production, project, and prototype:
rocket 4% ls -l
drwxr-xr-x 2 dan user 9 Nov 9 19:52 production/
drwxr-xr-x 2 dan user 76 Nov 9 19:50 project/
drwxr-xr-x 2 dan user 40 Nov 9 19:49 prototype/
The most-current project data needs to be put into the production
directory while maintaining permissions and recursively copying all the files
down the tree.
rocket 5% cp -pr project production
A symbolic link in the project directory points to the data in
the prototype directory. In a case such as this, you want to copy all the physical
data files that can be copied into the production directory. The copy with the
recursive permissions command works best for this scenario when there are embedded
symbolic links that it can follow.
rocket 6% ls -l project
-rw-r--r-- 1 dan user 66 Nov 9 19:49 file01
lrwxr-xr-x 1 dan user 10 Nov 9 19:49 prototype -> ../prototype
Inside the project directory are a file and a symbolic link that
point to more files in the prototype directory.
rocket 7% ls -l prototype
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh10180
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh120100
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh140960
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh193920
When all the data files in the project directory are copied, the
command treats the symbolic link as a directory and creates a subdirectory in
the production directory.
rocket 8% ls -l production
drwxr-xr-x 3 dan user 40 Nov 9 19:49 project/
Under the project directory is the transposed symbolic link that
is now a subdirectory called prototype.
rocket 9% ls -l production/project
-rw-r--r-- 1 dan user 66 Nov 9 19:49 file01
drwxr-xr-x 2 dan user 76 Nov 9 19:50 prototype/
All the files that were in the subdirectory that the link was
pointing to were copied into the new destination subdirectory.
rocket 10% ls -l production/prototype/project
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh10180
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh120100
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh140960
-rw-r--r-- 1 dan user 99 Nov 9 19:50 sh193920
Example Two: Copy with tar
Flavors: AT&T, BSD
Shells: All
Syntax:
tar -[cfx] - [file|dir]
The main uses for tar have always
been to archive data off to tape or archive a large number of files into one
file. It can also be used to copy data from one location to another. Most users
would use tar to archive the data into one file,
move the archived file to the destination, and then extract it back to its normal
state.
rocket 11% tar -cvf project.tar project
a project/ 0K
a project/file01 61K
a project/file02 3K
a project/file03 19K
rocket 12% ls -al project.tar
-rw-r--r-- 1 jim 87552 Nov 8 19:47 project.tar
rocket 13% mv project.tar /disk2
rocket 14% cd /disk2
rocket 15% tar -xvf project.tar
x projects/, 0 bytes, 0 tape blocks
x projects/file01, 62038 bytes, 122 tape blocks
x projects/file02, 2448 bytes, 5 tape blocks
x projects/file03, 19167 bytes, 38 tape blocks
rocket 16% rm project.tar
|
Note - Some flavors of tar
can display various results. This is one result; yours might differ.
|
The steps you read can be replaced by a single-line command that
pipes the archived data to the destination for extraction:
rocket 17% tar -cvf - project | (cd /disk2; tar -xvf - )
/disk2
a projects/ 0K
a projects/file01 61K
x projects/, 0 bytes, 0 tape blocks
x projects/file01, 62038 bytes, 122 tape blocks
a projects/file02 3K
a projects/file03 19K
x projects/file02, 2448 bytes, 5 tape blocks
x projects/file03, 19167 bytes, 38 tape blocks
If you look at this line closer, it tars
the file into a buffer ( -), changes to the destination
directory, and extracts the files from the buffer all at once. From the results
of the verbose ( v) option you can see how the files
are archived ( a) into the buffer and extracted
( x) into the destination directory. For extremely
large amounts of files that are being transferred, you might want to leave off
the verbose option to gain more speed in the transfer. The speed is gained by
not dedicating any time to displaying the activity to standard output.
You can create an alias that copies the current directory and
all its contents to a new destination and shortens the command entirely:
alias cptar 'tar cvf - . | ( cd \!* ; tar xvf - )'
|
Note - The tar
command cannot archive block devices and paths that exceed 256 characters
in length. The archive will continue, but those paths will be excluded.
|
Example Three: Copy with cpio
Flavors: AT&T, BSD
Shells: All
Syntax:
find dir -print | cpio -pudv destination
The cpio command is similar to tar
in some respects. It is, however, more cryptic in the commands but is still
powerful when used. Where the tar command finishes,
cpio keeps going. The cpio
command supports up to 1024-character pathnames and can continue if an error
is encountered while archiving.
Copying
files from one directory to another is done differently with cpio.
With cpio, data is piped or redirected through
it and extracted at the destination directory.
rocket 18% cd project
rocket 19% ls -l
-rw-r--r-- 1 jim staff 8192 Nov 8 16:51 file01
-rw-r--r-- 1 jim staff 28415 Nov 8 16:52 file02
-rw-r--r-- 1 jim staff 40960 Nov 8 16:52 file03
rocket 19% find . -print | cpio -pumdv /production/
/production/./file02
/production/./file01
/production/./file03
152 blocks
This command isn't that confusing when you really look at
it. The find command progresses through all the
files and subdirectories down the tree from the current directory and pipes
each one to cpio. If you break the cpio
command down by the arguments that are being passed to it, the command makes
a lot of sense. It reads from standard in ( p),
and then copies ( u) each file unconditionally.
While doing this, it retains all previous modification times ( m)
and creates any directories ( d) that are needed.
The verbose option ( v) lists all the filenames
that are being copied to the destination directory.
If there is a standard list of files, including their paths, that
need to be copied to a destination directory, the cat
command can be used to pipe data into the cpio file as well:
rocket 20% cat project.txt | cpio -pumdv /production
In this example, project.txt is a list of needed project files
that is copied to the /production directory.
rocket 21% vi project.txt
/project/file01.txt
/project/texture/tex03.dat
/project/pdf/ppp05.pdf
Reason
There are critical files that might need to retain their permissions
and time stamps but be moved to other filesystems on the local system.
Real World Experience
Using the tar and cpio
commands should become second nature to system administrators. Consider using
these commands when you deal with users' home directories, system files,
application files, database files, and program-development files.
There are versions of the tar command
that support error correction and exceed the 256 character path limitations.
They are available from GNU at their Web site (see "Other Resources,"
immediately following).
Other Resources
Man pages:
cpio, find,
ls, tar, whoami
World Wide Web:
GNU's Not UNIX site - http://www.gnu.org
© Copyright Macmillan USA. All rights
reserved.
Previous
Section