Stream Editor - sed

Running sed

The sed editor expects to read from the standard input and write to the standard output. You can provide a filename on the command line, but sed still writes the output to standard output. This means that sed does not change the file it edits. You can redirect the output to another file or to the original file, thereby changing that file.

By default, sed applies each editing command globally to all lines in the file. You can, however, use line addresses or patterns to restrict the lines to which the editing commands apply.

To use sed on the fly from the shell prompt, you can run it as follows:

somecommand | sed 'editcommand' | anothercommand

In this case, feed the output of somecommand to sed (sed applies editcommand to each line); sed’s output goes to anothercommand. For example, to see all the current processes that have gnome in their name, you can type the following command, which uses sed to edit the stream:

ps ax | sed -n /gnome/p | cut -c 28-80
/usr/bin/gnome-session
gnome-settings-daemon --oaf-activate-iid=OAFIID:GNOME
gnome-panel --sm-config-prefix /gnome-panel-9pyRb5/ -
gnome-terminal --sm-config-prefix /gnome-terminal-sHh
gnome-pty-helper
sed -n /gnome/p

In this case, each line in the output of the ps command is edited by sed, and a part of each line is extracted by using the cut command. Here, sed [i2]is invoked as follows:

sed -n '/gnome/p'

The -n option stops sed from printing each line to standard output. The '/gnome/p' part of the command line is an editing command—it means print each line containing the string gnome.

If you want to apply multiple sed commands, use the -e option to specify each editing command. For example, the following sed command takes the output of ps and deletes all lines that do not contain gnome (this is the edit command -e '/gnome/!d') and then deletes any line that contains the string sed (this is done by -e '/sed/d'):

ps ax | sed -e '/gnome/!d' -e '/sed/d' | cut -c 28-80
/usr/bin/gnome-session
gnome-settings-daemon --oaf-activate-iid=OAFIID:GNOME
gnome-panel --sm-config-prefix /gnome-panel-9pyRb5/ -
gnome-terminal --sm-config-prefix /gnome-terminal-sHh
gnome-pty-helper

Of course, you typically do not type long sed commands at the shell prompt. Instead, sed is used in shell scripts. You can also place sed commands in a file and edit a text file by running sed with the -f option, as follows:

sed –f sedscriptfile myfile

This command edits myfile by using the sed commands in sedscriptfile, and writes the lines to standard output. If you want to save the output in another file, redirect the output as follows:

sed –f sedscriptfile myfile > editedfile

Learning the Basics of sed Commands

All sed commands have the following general form:

[address][,address][!]command[arguments]

The parts shown in square brackets are optional. The command is an editing command (similar to those for the ed editor) that specifies the action sed takes on a line. The address specifies the lines to which the commands apply. The exclamation mark (!) applies the command to all lines that do not match the address. The arguments are needed by some editing commands, such as the w or r command, that read a file for which you have to provide the filename as an argument.

Secret

Understanding the address is the key to learning sed. A sed command can have zero, one, or two addresses. The addresses identify lines to which sed commands are to be applied. An address is a regular expression or a line number, or a symbol such as $ that refers to a line number. Use the following guidelines to understand how addresses control what gets edited:

• If there are no addresses, sed applies the command to each line. For example, if a script has the command s/UNIX/Linux/, sed changes every occurrence of UNIX to Linux.

• If there is only one address, sed applies the command to any lines matching the address. For example, 1d deletes the first line of the stream.

• If you specify two comma-separated addresses, sed applies the command to the first matching line and to all subsequent lines up to, and including, the line matching the second address. Thus, the following command deletes everything from line 26 to the last line in the file:

      26,$d

• If you add an exclamation mark (!) after an address, the command applies to all lines that do not match the address. For example, the command /gnome/!d deletes all lines that do not contain the string gnome.

  You can use curly braces to apply more than one editing command to an address. The syntax is as follows:

  [/pattern/][,/pattern]{
  command1
  command2
  ...other commands...
  }

  Note that the opening curly brace ({) must end a line, and the closing curly brace (}) must be on a line by itself. You should ensure that there are no blank spaces after the curly braces.

Sometimes you need to delete a line but to hold it in temporary storage and insert it elsewhere in the output stream. There are sed commands, such as h and g, for such operations. To use these commands, you need to understand the concept of pattern space and hold space. The pattern space is the buffer holding the current input line. The hold space is a separate set-aside buffer used for temporary storage. You can copy the contents of the hold space to the pattern space, and vice versa. For example, suppose that you want to extract all lines containing the string Item: and move them to a location later in the file (assume that the place is a line that has the string List of items:). You can do this with the following sed script:

/Item:/{
H
d
}
/List of items:/G  

The first four lines of the script apply the commands H and d to the lines that contain Item:. The H command appends the line to the hold space; the d command deletes it. Then the G command on the last line of the script inserts the contents of the hold space after the string List of Items:.

Because of limited space, this section does not show you all the sed commands in detail. However, Table 8-6 provides a summary of commonly used sed commands, grouped by function.

$a\
This line goes at the end of the file.\
Then this one.

1,50c\ 
<50 lines deleted>

/SKU 53055/i\ 
*** THE FOLLOWING ITEM IS OUT OF STOCK ***

/^SECTION/{
n
p
}

/^$/
{
N
/^\n$/D
}

/Order/{
h
s/Order \#[0-9]\{4\}$/We have shipped your order:/ 
p
x
}