Playing with file descriptors and redirection

File descriptors are integers that are associated with file input and output. They keep track of opened files. The best-known file descriptors are stdin, stdout, and stderr. We even can redirect the contents of one file descriptor to another. This recipe shows examples on how to manipulate and redirect with file descriptors.

Getting ready

While writing scripts we use standard input (stdin), standard output (stdout), and standard error (stderr) frequently. Redirection of an output to a file by filtering the contents is one of the essential things we need to perform. While a command outputs some text, it can be either an error or an output (nonerror) message. We cannot distinguish whether it is output text or an error text by just looking at it. However, we can handle them with file descriptors. We can extract text that is attached to a specific descriptor.

File descriptors are integers associated with an opened file or data stream. File descriptors 0, 1, and 2 are reserved as follows:

• 0: stdin (standard input)
• 1: stdout (standard output)
• 2: stderr (standard error)

How to do it...

1. Redirecting or saving output text to a file can be done as follows: $ echo "This is a sample text 1" > temp.txt

   This would store the echoed text in temp.txt by truncating the file, the contents will be emptied before writing.

2. To append text to a file, consider the following example: $ echo "This is sample text 2" >> temp.txt
3. You can view the contents of the file as follows: $ cat temp.txt This is sample text 1 This is sample text 2
4. Let us see what a standard error is and how you can redirect it. stderr messages are printed when commands output an error message. Consider the following example: $ ls + ls: cannot access +: No such file or directory

   Here + is an invalid argument and hence an error is returned.

   Tip

   
   

   Successful and unsuccessful commands

   When a command returns after an error, it returns a nonzero exit status. The command returns zero when it terminates after successful completion. The return status can be read from special variable $? (run echo $? immediately after the command execution statement to print the exit status).

   The following command prints the stderr text to the screen rather than to a file (and because there is no stdout output, out.txt will be empty):

   $ ls + > out.txt ls: cannot access +: No such file or directory

   In the following command, we redirect stderr to out.txt:

   $ ls + 2> out.txt # works

   You can redirect stderr exclusively to a file and stdout to another file as follows:

   $ cmd 2>stderr.txt 1>stdout.txt

   It is also possible to redirect stderr and stdout to a single file by converting stderr to stdout using this preferred method:

   $ cmd 2>&1 output.txt

   Or the alternate approach:

   $ cmd &> output.txt
5. Sometimes, the output may contain unnecessary information (such as debug messages). If you don't want the output terminal burdened with the stderr details then you should redirect the stderr output to /dev/null, which removes it completely. For example, consider that we have three files a1, a2, and a3. However, a1 does not have the read-write-execute permission for the user. When you need to print the contents of files starting with a, we use the cat command. Set up the test files as follows: $ echo a1 > a1 $ cp a1 a2 ; cp a2 a3; $ chmod 000 a1 #Deny all permissions

   While displaying contents of the files using wildcards (a*), it will show an error message for file a1 as it does not have the proper read permission:

   $ cat a* cat: a1: Permission denied a1 a1

   Here, cat: a1: Permission denied belongs to the stderr data. We can redirect the stderr data into a file, whereas stdout remains printed in the terminal. Consider the following code:

   $ cat a* 2> err.txt #stderr is redirected to err.txt a1 a1 $ cat err.txt cat: a1: Permission denied

   Take a look at the following code:

   $ cmd 2>/dev/null

   When redirection is performed for stderr or stdout, the redirected text flows into a file. As the text has already been redirected and has gone into the file, no text remains to flow to the next command through pipe (|), and it appears to the next set of command sequences through stdin.

6. However, there is a way to redirect data to a file, as well as provide a copy of redirected data as stdin for the next set of commands. This can be done using the tee command. For example, to print stdout in the terminal as well as redirect stdout into a file, the syntax for tee is as follows: command | tee FILE1 FILE2

   In the following code, the stdin data is received by the tee command. It writes a copy of stdout to the out.txt file and sends another copy as stdin for the next command. The cat -n command puts a line number for each line received from stdin and writes it into stdout:

   $ cat a* | tee out.txt | cat -n cat: a1: Permission denied 1a1 2a1

   Examine the contents of out.txt as follows:

   $ cat out.txt a1 a1

   Note that cat: a1: Permission denied does not appear because it belongs to stderr. The tee command can read from stdin only.

   By default, the tee command overwrites the file, but it can be used with appended options by providing the -a option, for example, $ cat a* | tee -a out.txt | cat -n.

   Commands appear with arguments in the format: command FILE1 FILE2 . or simply command FILE.

7. We can use stdin as a command argument. It can be done by using - as the filename argument for the command as follows: $ cmd1 | cmd2 | cmd -

   For example:

   $ echo who is this | tee - who is this who is this

   Alternately, we can use /dev/stdin as the output filename to use stdin.

   Similarly, use /dev/stderr for standard error and /dev/stdout for standard output. These are special device files that correspond to stdin, stderr, and stdout.

How it works...

For output redirection, > and >> operators are different. Both of them redirect text to a file, but the first one empties the file and then writes to it, whereas the later one adds the output to the end of the existing file.

When we use a redirection operator, the output won't print in the terminal but it is directed to a file. When redirection operators are used, by default, they operate on standard output. To explicitly take a specific file descriptor, you must prefix the descriptor number to the operator.

> is equivalent to 1> and similarly it applies for >> (equivalent to 1>>).

When working with errors, the stderr output is dumped to the /dev/null file. ./dev/null is a special device file where any data received by the file is discarded. The null device is often known as a black hole as all the data that goes into it is lost forever.

There's more...

A command that reads stdin for input can receive data in multiple ways. Also, it is possible to specify file...