1. Advanced Programming in the UNIX Environment
By W.Richard Stevens
2. UNIX Programmer's Manual, Section 2.
Create a directory called [file] and change into that directory
$ mkdir $HOME/file
$ cd $HOME/file
Bring [ file1.txt ] in [file] directory. Carry out the experiments in
that directory.
System calls:
UNIX provides a number of entry points directly into the
kernel. These are called system calls. Through these system calls, a
program requests services from the kernel.
The system calls [read], [write], [umask], [creat], [open], [lseek],
[close], [exit], [dup], [dup2] and [fcntl] are introduced in this
experiment.
The system calls are documented traditionally in section 2 of UNIX
Programmer's Manual. The contents of the directory [/usr/share/man/
man2] can be checked to find out the system calls that are documented
in section 2 of UNIX Programmer's Manual.
Example: The following compound command, lists the files in the [man2]
directory. The output of the first command is is used as the
input of the [wc -l] command. The [wc -l] command counts the
number of newlines.
$ ls -l /usr/share/man/man2 | wc -l
Output : Number of lines -> in [/usr/share/man/man2] directory.
Inference : There might be that many system calls described in section
2 of UNIX Programmer's Manual.
In UNIX systems, each system call has a function of the same name in
standard C library regardless of the actual implementation technique
used on any given system (with a given processor) to invoke the system
call. The user process calls this function, using standard C calling
sequence. The function may put one or more C arguments in general
purpose registers and then execute some machine instruction that
generates a particular software interrupt in the kernel. User programs
are insulated from the processor specific details with the C function
calls. For the user program the system calls are C functions calls.
Write the following C program which uses [write(2)] system call.
//-------------------------------------------
// file-name f1a.c
#include
int main(void)
{
write( STDOUT_FILENO, "Hello CEMK\n", 11 );
return 0;
}
//-------------------------------------------
Compile the program
$ gcc -Wall ./f1a.c -o ./onea
The option [ -Wall ] should be pronounced as "Warning all".
Execute the program
$ ./onea
Login in another terminal and read manual of [write(2)] system call
$ man 2 write
From the manual we find out that the file [ /usr/include/unistd.h ] is
to be included in a program, if we want to use [ write ] system call.
This header was included in [f1a.c] by consulting the manual.
The [ write ] system call takes three arguments:
1st argument: The file descriptor in which bytes are written.
2nd argument: The buffer from which bytes are taken.
3rd argument: Number of bytes to be written to the file descriptor.
In [f1a.c] the 1st argument was STDOUT_FILENO. This is the descriptor
for the file called standard output. The usual numerical value of this
symbolic constant is 1. The CRT screen is the default standard output.
The 2nd argument was the buffer. Contents of the buffer was the string
"Hello CEMK\n". The string was composed of 11 bytes including the new-
line.
The 3rd argument was the size of the 2nd argument.
The [ write ] system call returns the number of bytes, written on the
file descriptor, if the call is successful. [ssize_t] is the data type
of the returned value.
The returned value of the [ write ] system call is printed in the next
program.
-----------------------------------------------------------
// file-name f1b.c
#include
int main(void)
{
ssize_t i;
i = write( STDOUT_FILENO, "Hello CEMK\n", 11 );
printf( "%u bytes were written on file descriptor %u \n",
i, STDOUT_FILENO );
return 0;
}
-----------------------------------------------------------
This program uses the library function [ printf(3) ] to print value of
the symbolic constant STDOUT_FILENO and the number of bytes written on
this file descriptor.
Try to compile the the program
$ gcc -Wall ./f1b.c -o ./oneb
Compiler should issue warning message as needed header for using the
[printf(3)] function was not included. Read the manual of [printf(3)]
library function to find out the header to be included...
$ man 3 printf
The needed header [/usr/include/stdio.h] was included in the [f1c.c].
-----------------------------------------------------------
// file-name f1c.c
#include
#include
int main(void)
{
ssize_t i;
i = write( STDOUT_FILENO, "Hello CEMK\n", 11 );
printf( "%u bytes were written on file descriptor %u \n",
i, STDOUT_FILENO );
return 0;
}
-----------------------------------------------------------
Compile and execute the binary
$ gcc -Wall ./f1c.c -o ./onec
$ ./onec
When the program is run in the shell, OS starts a new process for this
program and allots a Process IDentification ( PID ) number to the
process, among other things.
OS also opens three frequently needed files for this process.
The three descriptors are:
STDIN_FILENO
STDOUT_FILENO and
STDERR_FILENO
The process may use these file descriptors.
OS also creates a directory [/proc/pid-of-the-process] to keep various
information about the process in this directory. This directory is
removed by OS, when the program terminates. An endless loop is used at
the end of the next program, to prevent the program from terminating.
The [ /proc/pid-of-the-process ] directory may be examined, during the
lifetime of the process.
A process may get the value of own PID using [getpid(2)] system call.
Note: [getpid(2)] system call always succeeds !
-----------------------------------------------------------
// file-name f1d.c
#include
#include
int main(void)
{
ssize_t i;
i = write( STDOUT_FILENO, "Hello CEMK\n", 11 );
printf( "%u bytes were written on file descriptor %u \n",
i, STDOUT_FILENO );
// process identification number of this process
printf("PID of this process = %u \n", getpid() );
while( 1 ) { sleep(1); } // an endless loop
return 0;
}
-----------------------------------------------------------
The program is compiled and run...
$ gcc -Wall ./f1d.c -o ./oned
$ ./oned
Value-of-PID = _______
Due to the endless loop, this terminal would be tied up. Use another
terminal for the next command.
Note: You may run the program in background and continue to use this
terminal.
$ ls -l /proc/Value-of-PID/fd
[fd] stands for file descriptors.
The process should have three ( 0, 1 and 2 ) opened file descriptors,
which stand for the following three files:
Name of file Symbolic constant Numerical value in this PC
of the descriptors
standard input STDIN_FILENO 0
standard output STDOUT_FILENO 1
standard error STDERR_FILENO 2
A process should close all open file descriptors when they are not to
be used anymore, by calling [ close(2) ] system call. All open file
descriptors can be closed by calling [ exit(3) ] library function. If
the file descriptors are not closed with [close(2)] or [exit(3)] and
the process exists, OS closes all its open file descriptors.
Go back to the previous terminal and terminate the program [oned] with
CTRL-C.
The manual of [ write(2) ] states that if the call fails due to any
reason, the returned value becomes -1. This is used to detect failure
of the call. When a call fails, the program should exit, instead of
giving some wrong result. To simulate error condition, try to write to
a non-existent file descriptor 4.
-----------------------------------------------------------
// file-name f1e.c
#include
#include
int main(void)
{
ssize_t i;
// trying to write to an invalid file descriptor
i = write( 4, "Hello CEMK\n", 11 );
if( i == -1 )
{
printf("ERROR !!\n");
exit(1);
}
printf( "%u bytes were written on file descriptor %u \n",
i, STDOUT_FILENO );
return 0;
}
-----------------------------------------------------------
Compile and execute the binary
$ gcc -Wall ./f1e.c -o ./onee
$ ./onee
The error condition was indicated in the output but what caused the
[ write(2) ] call to fail, was not indicated in the program output. We
have done error checking for one function call. Thus we know which
function call failed. In a large program, where error checking is done
on a number of function calls, it would be difficult to know which
function call caused the error, with such simple error checking.
[strace] utility:
This shell command is a useful diagnostic, debugging
and instructional tool. This would be used throughout this course to
understand certain features of C programs written by you.
We run the above executable program with [strace]
$ strace ./onee
Look for a line, similar to the following line in the output
write(4, "Hello CEMK\n", 11) = -1 EBADF (Bad file descriptor)
This line indicates that writing on file descriptor 4 failed with exit
status of -1 and the error was EBADF ( Bad file descriptor ).
The manual for [write(2)] call documents EBADF as one of the possible
errors.
Section 3 of UNIX Programmer's Manual documents the general purpose C
library functions, available to the programmer.
These library functions are not entry points in the kernel but they
may invoke one or more system calls to the kernel.
Look for a line, similar to the following line, in the output of
[ strace ] command.
write(1, "ERROR !!\n", 9ERROR !!) = 9
This line indicates that [ printf(3) ] invoked [write(2)] system call
and used STDOUT_FILENO to print on CRT screen. Nine bytes were written
in standard output.
The EBADF error is a symbolic constant and it has a numerical value.
Record the numerical value of EBADF, from the output of the following
command.
$ less /usr/include/asm/errno.h
EBADF = ____
In the next program, the cause of the failure of the call is printed
in human-readable form. Library function [perror(3)] is used for this.
-----------------------------------------------------------
// file-name f1f.c
#include
#include
int main(void)
{
ssize_t i;
i = write( 4, "Hello CEMK\n", 11 );
if( i == -1 )
{
printf("ERROR !!\n");
perror("write-call");
exit(1);
}
printf( "%u bytes were written on file descriptor %u \n",
i, STDOUT_FILENO );
return 0;
}
-----------------------------------------------------------
The program was compiled and run...
$ gcc -Wall ./f1f.c -o ./onef
$ ./onef
output:__________________________________________
__________________________________________
The program was run with [strace]
$ strace ./onef
Lines similar to the following lines were checked:
dup(2) = 3
.
.
write(3, "write-call: Bad file descriptor\n", 32
write-call: Bad file descriptor) = 32
The above lines show that a duplicate file descriptor, for standard
error( 2 ) was created by [ dup(2) ] system call. This duplicate file
descriptor was three(3). After the [dup(2)] call, either 2 or 3 could
be used as the standard error. [perror(3)] library function used file
descriptor 3 to write the error message on screen.
When programs communicate with each other ( IPC ), error messages in
human readable form are not useful. Programs deal conveniently with
error numbers.
The next program shows how an error number and numeric value of the
symbolic constant EBADF may be printed.
-----------------------------------------------------------
// file-name f1g.c
#include
#include
#include
extern int errno;
int main(void)
{
ssize_t i;
printf("numerical value of EBADF = %u \n", EBADF );
errno = 0;
printf("before [write] call: errno = %u\n", errno );
i = write( 4, "Hello CEMK\n", 11 );
if( i == -1 )
{
printf("ERROR !!\n");
printf("after [write] call: errno = %u \n", errno );
exit(1);
}
printf("bytes written on %u = %u \n", STDOUT_FILENO, i );
return 0;
}
-----------------------------------------------------------
$ gcc -Wall ./f1g.c -o ./oneg
$ ./oneg
Usage of file STDIN_FILENO is shown in the next program. Read manual
of [read(2)] system call.
This call takes three arguments:
1st argument: file descriptor from which bytes are read.
2nd argument: buffer in which the read bytes are stored.
3rd argument: number bytes to be read.
Headers are not included in this program. Needed headers are to be
included by reading manual pages of the function calls.
-----------------------------------------------------------
// file-name f2.c
#define BUFFERSIZE 100
int main( void )
{
ssize_t i,j;
char buffer[BUFFERSIZE];
// a prompt is generated
i = write( STDOUT_FILENO, "Please enter a string ", 22 );
i = read( STDIN_FILENO, buffer, BUFFERSIZE );
if( i == -1 )
{
perror("read");
exit(1);
}
printf("%u bytes were entered at the keyboard \n", i );
// the contents of the buffer are printed on screen
j = write( STDOUT_FILENO, buffer, i );
if( j == -1 ) { perror("write"); exit(2); }
printf("%u bytes were printed on the screen \n", j );
return 0;
} // end of main
-----------------------------------------------------------
Try to compile the program...
$ gcc -Wall ./f2.c -o ./two
Remove the errors by including the appropriate headers. The following
calls were used in the program:
write
read
perror
exit
printf
The manuals of the calls are to be consulted to find the headers.
$ man 2 write
$ man 2 read
$ man 3 perror
$ man 3 exit
$ man 3 printf
Run the program after correct compilation.
$ ./two
For an application program to compile and run correctly on different
UNIX implementation without modification (portability), the operating
system interface must follow a standard. POSIX ( Portable Operating
System Interface) is such a standard.
Example: By convention the UNIX shell associates file descriptors
0 ( Zero) with standard input
1 with standard output
2 with standard error
The magic numbers 0, 1 and 2 are replaced in POSIX standard with
symbolic constants STDIN_FILENO, STDOUT_FILENO and STDERR_FILENO
respectively. The header file <> includes the above three,
among others. In programs intended to be portable, symbolic constants
must be used instead of magic numbers.
When a process starts, OS makes available three file descriptors to
the programmer. Input and output, using these file descriptors, were
done in the previous examples. If any other file is to be accessed
from a C program, that file is to be "opened". Read the manual of
[ open(2) ] system call.
$ man 2 open
This call, if successful returns a file descriptor. This would be the
lowest file descriptor not currently open for the process.
The next program intends to read an existing file [/etc/passwd].
------------------------------------------------------------
// file-name f3.c
#include
#include
#include
#include
#include
#define BUFFERSIZE 128
int main(void)
{
int m;
char buffer[BUFFERSIZE];
ssize_t i,j;
// the file is tried to be opened for reading
m = open( "/etc/passwd", O_RDONLY );
if( m == -1 ) { perror("open"); exit(1); }
printf("value of the new file descriptor = %u \n", m );
// 80 bytes are tried to be read from the file descriptor.
// The bytes are saved in the buffer called "buffer"
i = read( m, buffer, 80 );
if( i == -1 ) { perror("read"); exit(2); }
printf("%u bytes were read from descriptor %u \n", i, m );
j = write( STDOUT_FILENO, buffer, i );
if( j == -1 ) { perror("write"); exit(3); }
printf("\n%u bytes were written on screen \n", j );
printf("PID = %u \n", getpid() );
// an endless loop to check file descriptors
while(1) { sleep(1); }
return 0;
} // end of main
------------------------------------------------------------
Compile and run the program in background
$ gcc -Wall ./f3.c -o ./three
$ ./three &
Check open file descriptors of the program
$ ls -l /proc/value-of-PID/fd
Record the line corresponding to the new file descriptor
Bring the process in foreground and terminate it with CTRL-C
$ fg job-number
CTRL-C
Create a file [kk] using shell command and redirection operator
$ echo "My name is your-name" > ./kk
Examine contents of the file
$ less ./kk
Next program opens [ /etc/passwd ] file in read mode and opens file
[./kk] in append node. A few bytes are read from [/etc/passwd] file
and are appended to the file [./kk].
------------------------------------------------------------
// file-name f4.c
#include
#include
#include
#include
#include
#define BUFFERSIZE 512
int main(void)
{
int m,n;
char buffer[BUFFERSIZE];
ssize_t i,j;
// file [/etc/passwd] is tried to be opened for reading.
m = open("/etc/passwd", O_RDONLY);
if( m == -1 ) { perror("1st-open"); exit(1); }
printf("descriptor of /etc/passwd = %u \n", m );
// file [./kk] is tried to be opened in append mode
n = open( "./kk", O_WRONLY | O_APPEND );
if( n == -1 ) { perror("2nd-open"); exit(2); }
printf("descriptor of ./kk = %u \n", n );
// 100 bytes are read from [/etc/passwd] file
i = read( m, buffer, 100 );
if( i == -1 ) { perror("read"); exit(3); }
printf("%u bytes were read from descriptor %u \n", i, m );
// write contents of buffer in descriptor of file [./kk]
j = write( n, buffer, i );
if( j == -1 ) { perror("write"); exit(3); }
printf("%u bytes were written on descriptor %u \n", j,n );
return 0;
} // end of main
------------------------------------------------------------
Compile and run the program
$ gcc -Wall ./f4.c -o ./four
$ ./four
Check the contents of the file [./kk].
Were 100 bytes appended to it ? ( Y / n )
When a file is opened, a file position pointer is used to point to the
appropriate location of the file. If a file is opened in append mode,
the file position pointer (file offset) is set at the end of the file.
instead of on the first byte of the file. If [four] is run again, 100
bytes would be appended to it again.
Creation of files using system calls:
Files can be created from the
command line using various editors, [touch] command, redirection etc
etc. Files can also be created from within a C program using system
calls [creat] and [open]. You must read manual of [creat(2)].
Read the manual of [umask(2)] system call.
In the next example a file [x1] is created, using [ creat(2) ] system
call. The file is tried to be created with read, write and execute
permissions for owner, group and others. But due to the default value
of the file creation mask, the the file may not be created with the
requested permissions. As an example, if the value of umask is 0022,
the execute permissions for group and others would be turned off.
Before creation of the file [./x2] the value of the file creation mask
was set to 0777. This value would turn off all permissions.
Before creation of the file [./x3] the value of the file creation mask
was set to 0000. This value would allow creation of a file with all
the requested permissions.
The program also shows the use of [unlink(2)] system call.
-----------------------------------------------------------------
// file-name f5.c
#include
#include
#include
#include
#include
#define PERMISSIONS ( S_IRWXU | S_IRWXG | S_IRWXO )
// file owner, group and others have read,
// write and execute permissions
int main (void)
{
int m;
// permissions in octal
printf("requested permissions of files = %o \n", PERMISSIONS );
//remove file [./x1] if it exists from a previous run
m = unlink("./x1");
if( m == -1 ) { perror("unlink-1"); }
// Create a file [x1] with PERMISSIONS defined above and using
// the default file creation mask
m = creat( "./x1", PERMISSIONS );
if( m == -1 ) { perror("creat-x1"); exit(1); }
// remove file [./x2] if it exists from a previous run
m = unlink("./x2");
if( m == -1 ) { perror("unlink-2"); }
// change the file creation mask to 0777
umask( 0777 );
// Create a file [x2] with PERMISSIONS defined above and using
// the new file creation mask
m = creat( "./x2", PERMISSIONS );
if( m == -1 ) { perror("creat-x2"); exit(2); }
// remove file [./x3] if it exists from a previous run
m = unlink("./x3");
if( m == -1 ) { perror("unlink-3"); }
// clear the file creation mask
umask( 0000 );
// create a file [x3] with PERMISSIONS defined above and using
// the cleared file creation mask
m = creat( "./x3", PERMISSIONS );
if( m == -1 ) { perror("creat-x3"); exit(3); }
return 0;
}
-----------------------------------------------------------------
Compile and run the program
$ gcc -Wall ./f5.c -o ./five
$ ./five
The expected permissions of the files [x2] and [x3] are shown below:
---------- x2
-rwxrwxrwx x3
Record the permission of the file [x1].
Save the following program as [ f6.c ].
------------------------------------------------------------------
// file-name f6.c
#include
#include
#include
#include
#include
int main (void)
{
int m;
umask(0000); // file creation mask is cleared
// remove file [./x4] if it exists from a previous run
m = unlink("./x4");
if( m == -1 ) perror("unlink");
// create file [x4] with read and write permissions for user,
// read permissions for group and others
m = creat("./x4", S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH );
if( m == -1 ) { perror("creat"); exit(1); }
printf("The file descriptor of [x4] is %u\n", m );
write( m, "This is first line\n", 19 );
// file position pointer is moved 5 places from current position
// The intention is to keep five byte hole after the first line
lseek( m, 5, SEEK_CUR );
write( m, "This is the second line\n", 24 );
close(m);
return 0;
}
------------------------------------------------------------------
The new file [x4] was created with read and write permission for user
( S_IRUSR | S_IWUSR ), read permission for group ( S_IRGRP ) and read
permission for others ( S_IROTH ). On success, [ creat(2) ] returns a
new file descriptor.
Another system call [lseek] was used to re-position the offset of the
file descriptor ( file position pointer ). Then another write was
performed on the file.
Compile and run the program
$ gcc -Wall ./f6.c -o ./six
$ ./six
Examine the created file [x4] with [od] command
$ od -a ./x4
A gap of 5 bytes should be present between the two lines of text.
[ open(2) ] system call can also be used to create a new file, using
appropriate flags.
--------------------------------------------------------------------
// file-name f7.c
#include
#include
#include
#include
#include
#define flags ( O_RDWR | O_CREAT | O_TRUNC )
#define mode ( S_IRWXU | S_IRWXG | S_IRWXO )
int main (void)
{
int fd1,fd2,fd3,fd4 ; // file descriptors
int m;
ssize_t i;
// remove x5 if it exists from a previous run
m = unlink("./x5");
if( m == -1 ) perror("unlink-call");
umask(0000); // clear file creation mask
// create a file [x5]
fd1 = open("./x5", flags, mode);
if( fd1 == -1 ) { perror("open"); exit(1); }
printf("open : fd of x5 = %u \n", fd1 );
// write a line of text in fd1
write( fd1, "First line of x5\n", 17);
// move the file offset 30 bytes from start of file
lseek( fd1,30,SEEK_SET);
// create a duplicate file descriptor of [fd1]. [fd2] would be
// the lowest-numbered unused descriptor
fd2 = dup(fd1);
if( fd2 == -1 ) { perror("dup"); exit(2); }
printf("dup : fd of x5 = %u \n", fd2 );
// write a line of text in fd2
write( fd2, "Second line of x5\n", 18 );
// create another duplicate file descriptor of [fd1]. Value of
// fd3 would be 6, if the call is successful
fd3=dup2(fd1,6);
if( fd3 == -1 ) { perror("dup2"); exit(3); }
printf("dup2 : fd of x5 = %u\n", fd3 );
// write a line of text in fd3
write( fd3, "Third line of x5\n", 17 );
// create yet another duplicate file descriptor of [fd1]. Value of
// [fd4] would be lowest-numbered unused descriptor greater than
// zero
fd4 = fcntl( fd1, F_DUPFD, 0 );
if( fd4 == -1 ) { perror("fcntl"); exit(4); }
printf("fcntl: fd of x5 = %u\n", fd4 );
// write a line of text in fd4
write( fd4, "Fourth line of x5\n", 18 );
for( m = 0; m <= 6; m++ )
{
printf("writing on descriptor %u\n", m );
i = write( m, "xxxx\n", 5 );
if( i == -1 ) perror("write-call");
}
close(fd1);
close(fd2);
close(fd3);
close(fd4);
return 0;
}
--------------------------------------------------------------------
File creation mask was cleared to create a new file with the requested
permissions. A file [x5] was created with [open(2)] system call. Read,
write and execute permissions for owner, group and others were
requested.
Then one line was written into it using descriptor [ fd1 ]. The file
descriptor offset advanced to the end of this line. To change this,
[lseek] system call was called to set the file descriptor offset 30
bytes from the start of the file ( NOT from the current position ).
With [dup] system call a copy of [fd1] was created. It was called
[fd2].
[fd1] and [fd2] referred to the same file [x5].
Access mode of [fd2] was the same as that of [fd1].
[fd1] and [fd2] shared the same file offset ( file position pointer ).
Second line was written to the file using file descriptor [fd2].
[dup2] system call makes [fd3] the copy of fd1. 6 was requested as the
value of [fd3]. [fd3] referred also to file [x5].
Third line was written to the file using file descriptor [fd3].
The [fcntl] system call [ fd4 = fcntl( fd1, F_DUPFD, 0 ) ], finds the
lowest numbered file descriptor greater than or equal to zero and make
it a duplicate of [ fd1 ]. That will be [ fd4 ] if the call succeeds.
[fd4] referred also to file [x5].
Fourth line was written to the file using file descriptor [fd4].
After the [fcntl(2)] call four file descriptors for a single file [x5]
were open. Then the string "xxxx\n" was written using file descriptors
zero to 6, in a loop. This string was written four times into the file
for the four file descriptors.
Compile and run the program
$ gcc -Wall ./f7.c -o ./seven
$ ./seven
Record the following from the output:
open : fd of x5 = __
dup : fd of x5 = __
dup2 : fd of x5 = __
fcntl: fd of x5 = __
Examine the file [x5]
$ od -c ./x5
-------------------------------
Assignment-1.
(a) Execute the following command and fill up the blanks.
$ cat /usr/include/unistd.h | grep _FILENO
#define STDIN_FILENO __ /* Standard input. */
#define STDOUT_FILENO __ /* Standard output. */
#define STDERR_FILENO __ /* Standard error output. */
rite the contents of the buffer in file y1.
