clu2's notes: Notes on sed (stream editor)

sed one-liners

For the impatient.

More examples here.

Converting sed scripts to Perl scripts with s2p

See here

How does sed work ?

sed works in execution cycles. Cycle continues till end of file/input is reached. In each cycle, sed will

Read an entire line
Remove the terminating newline and then overwrite the pattern space (a kind of buffer) with it.
Run all commands (whose address match/do not match that line) in sequence on the pattern space.
Unless -n command-line option is in effect, print the pattern space (to stdout) with the newline added back.

In addition to pattern space, sed has another buffer called hold space, which is a placeholder for contents of the pattern space for subsequent retrieval. Initially, hold space is empty.

Invoking sed

sed -i -e "script" [list of files]	Run the `script` on `list of files`. The contents of `list of files` will be modified after the run.
sed -i -e "script1" -e "script2" ... [list of files]	Like above, but run multiple scripts on `list of files`.
sed -i -n -e "script" [list of files]	Similar to above, but only used when the `script` uses `p` (print) command (see below)
sed -e "script" < oldFile [> newFile]	Run the `script` on `oldFile` and redirect the result to `newFile`
sed -f scriptFile [list of files]	Run the `scriptFile` on `list of files`

Command syntax

A sed script consists a series of commands separated by semi-colon ;

A command has the form

[address1[,address2]] [!] command [arguments]

where the address1[,address2] specifies the range of lines the command will operate on. If they are omitted, then it means the entire input. If only address1 is present, the command will only operate on that address. (Some commands accept only zero or one address)

The optional exclamation ! means the command be run if the address (or address-range) does not match.

Address format

address can take the following formats:

Format	Meaning
`number`	The `number`-th line.
`$`	The last line.
`first~step`	Every `step`-th line starting with `first`-th line.
`/regex/`	Lines which match the regular expression `regex`.

See here for more details.

Commands

Common commands

Command	Effect
s/`regex/[replacement]/[flags]`	Replace the first occurrence of `regex` with `replacement`. `regex` is a sed regular expression. `replacement` can contain `\1`, `\2`, ... , `\9` to refer to matching groups (like `$1`, `$2`, ... in Perl) `flags` can be g: Replace all (global) non-overlapping occurrence of `regex`. n: Replace the n-th occurrence of `regex`. p: Print the pattern space if a succesful substitution takes place. w fileName: Write the pattern space to file named fileName if a succesful substitution takes place.
y/`srcChars/destChars/`	Transliterate `srcChars` with `destChars` (so they must have the same length.) It is like `tr` command.
p	Print the pattern space (with the newline added to the end). This command is usually only used in conjunction with the `-n` command-line option.
P	Print the pattern space up to the first embedded newline
n	Print the pattern space (unless `-n` command-line option is in effect), and overwrite the pattern space with the next line of input.
N	Append the next line of input to pattern space. The former and new contents are separated by a newline.
d	Delete the pattern space and start the next cycle (so nothing will be printed)
D	If there is a newline in pattern space, delete the pattern space up to the first embedded newline and start the next cycle (so nothing will be printed). Skip reading from the input if pattern space is non-empty. If there is no newline found, it behaves like d command.
=	(GNU sed only) Print the current line number (with the newline added to the end)
`l` [n]	Print the pattern space (with the newline added to the end) with special handling of unprintable characters. n specifies the desired line-wrap length (GNU sed only).

Hold space manipulation

Command	Effect
G	Append holding space to pattern space. The former and new contents are separated by a newline.
H	Append pattern space to holding space. The former and new contents are separated by a newline.
g	Overwrite pattern space with holding space
h	Overwrite holding space with pattern space
x	Exchange holding space and pattern space

Control flow

Command	Effect
: label	Define a label
b [label]	Jump to label or to the end of script if label is omitted.
t [label]	If s/// succeeds, jump to label or to the end of script if label is omitted.
T [label]	If s/// fails, jump to label or to the end of script if label is omitted.
q	Exit sed without any further input processing. This command only accepts one address.