Sed command in unix and Linux examples – How to use

Sed command in UNIX and Linux examples - How to use
Sed command in UNIX and Linux examples - How to use

 This tutorial is divided into five parts which includes.

  1. Sed command in UNIX and Linux examples introduction
    Sed with &
    Sed with s
    Sed with g
    Sed with d
    Delete ranges
    Sed with I
    Sed with e
    sed with = sign
    Sed  n option for not printing


  2. Basic Sed command in UNIX and Linux to replace string
    Sed replace string
    Sed replace string from nth pattern
    Replace all pattern
    Sed replace all pattern from nth pattern
    Replace string in a particular line
    Using other delimiter instead of (/)

  3. Advance Sed command in unix to search and replace string in Linux and Unix
    Adding extra character
    To change the position of character with sed
    Making changes to the string with \1, \2, \3
    Duplicating a line
    How to replace string on a particular line
    Using $ with the sed command
    How to delete line by sed
    To search any string in a file
    Sed to print duplicate lines
    How to run multiple sed command in unix
    Sed to print line which doesn't match
    Replacing a line with a new line by sed
    Sed command to add new line
    Transforming case of letters
    Removing blank lines
    Sed script
    How to replace a line starting with a space into a new line
    Sed regex examples
    Sed hold buffer and pattern buffer
    How to join two lines

      4. Man sed (Man page of sed command)
           Sed f example
           Sed follow symlinks
           Sed i option example
           Sed binary mode (--binary)
           Sed line wrap
           Sed unbuffered (-u)
           Sed --separate (-s)
           Sed with c\
           Change line with sed c
           Sed append (a\)
           Sed q command for exit
           Sed w flag
           Using flag y
           Sed branching (Labels)
     
       5. Tricks of sed command in UNIX and Linux
           Numbering each line of your file
           Counting lines
           Print lines having 20 or more than 20 characters in a file
           Delete all blank spaces from a file
           Print only valid email address from file
           Using alphabet character class keyword
           Sed & (Ampersand) trick


You may also like to read our one more article UNIX Sed tutorial written for Linux administrators for their daily stream editing tasks.

Also Download our FREE eBook on Linux and UNIX shell scripting

Sed command in UNIX & Linux introduction

  • Sed command is commonly used to replace string in Unix or Linux based OS. That is why it is more commonly used as 'sed replace'.
  • Sed is a stream editor in Unix. It is used to parse and transforms texts.
  • Sed was developed in the year 1973.
  • Sed uses compact programming language, it was built for command line processing.
  • Sed is also used to make programs which can change files.
  • It is based on two very old editors one is ed (editor) and another is qed (quick editor)

Sed command options

Before going deep into basic and advance of sed command in UNIX and Linux, It is salient to know about the options and argument you can apply with sed command. Some of the common sed examples with options are give below. By using the below sed examples you will get the basic idea of using sed in everyday stream editing.

Sed with &

The option & is used to append to the searched pattern. The syntax to use '&' is.

sed 's/search_pattern/& append_pattern/' file.txt

The below sed command will find the search pattern '123' and append '456' to it.

# sed 's/123/&456/' file.txt 

Now, suppose you are not sure of what pattern exactly you are looking for. You only remembers the first and last character of the string.  In that case you have to use special character along with '&'. Look at the below example where I have used '&' to append a new word.

# sed 's/u*x/& and Linux/' file.txt
Output: Learn how to use sed command in unix and Linux

Sed with s for substitution

S stands for substitution. It replaces the searched string with the new string. Although 's' has various roles with sed but most commonly it is used for substitution.

sed 's/searched_pattern/replaced_pattern/' file.txt

The below command will replace the string 'ABC' with 'ZYX'. In the given sed command, sed replaces only the first occurrence of the pattern in each line.

# sed 's/ABC/ZYX/' file.txt

Although the above command replaces 'ABC' with 'ZYZ' but the change is not permanent. For the permanent change you need to print the output to another file like given below.

# sed 's/ABC/ZYX/' file.txt > file2.txt

or you can use it another way just for testing purpose.

$ echo day | sed s/ABC/XYZ/
XYZ

If you have noticed, I haven't used quotes in the above sed command examples. But you are required to use quotes if you have meta-character in your command. Meta-characters has special meaning to a computer program. Some examples of meta-character are  brackets ([]), dollar sign ($), pipe (|), period (.), etc.

Sed tutorial for linux administrator

Sed with g

If we ask sed to replace the pattern with a new one, it only replaces the first occurrence of the searched patter. But some times just changing only the first occurrence is not enough. You have to change all words in every line.  g works as global replacement when used with sed command. The following command will replace all the searched pattern with the replaced pattern.

sed 's/searched_pattern/replaced_pattern/'g file.txt
Example:
# cat file.txt
grep and awk tool can be used for searching patterns.
grep and awk can used in UNIX & Linux.

Now, I am going to use 'g' to replace the whole searched pattern in the file.

# sed 's/grep/sed/'g file.txt
sed and awk tool can be used for searching patterns.
sed and awk can used in UNIX & Linux.

 

Sed with d

With more and more use of sed you will  find out that some times deleting lines is not that simple. You may have to delete lines matching specific  character or lines which are only spaces. The Option 'd' with sed  is used to delete lines from files.

The following command will delete all lines from file.

# sed 'd' file.txt

This will delete line number 2 from file.

# sed '2d' file.txt

Similarly, to delete 3rd and fourth line following commands can be used.

# sed '3d' file.txt
# sed '4d' file.txt

One of the practical example of sed d command is deleting the commented lines from configuration files. Let's take an example of deleting commented lines from 'httpd.conf' file. commented lines starts with # symbol.

# sed '/^#/ d' httpd.conf

Similarly, you can use the command differently to remove blank spaces from files.

# sed '/^$/d' file1.txt

Always remember to store the output to another file use the redirection operator.

# sed '/^$/d' file1.txt > savedfile.txt

Delete range of lines

It's not always that you have to use sed to delete single line all the time, you may have to delete range of lines. You can declare the range of lines to be deleted by declaring it in the command . For example to delete lines from first to third use sed the way it's given below.

$ sed '1, 3d' file.txt

However, you cannot delete lines in sed in a reverse order, because sed doesn't work in reverse way. For example the following sed command will delete only the third line in a file and leave the rest.

# sed '3, 1d' myfile.txt.

The above examples when you execute only print the output on the terminal, there is no change happen inside the file. Rather than just printing the line on the terminal, use '-i' to delete the line completely.

# sed -i '3, 1d' your_filename.

1~4d or 2~3d

The below sed command will delete the first line in a file, move on to the 4th line, then delete the fifth line in the file.

$ sed '1~4d' file.txt

Similarly, in this sed example, sed will delete the second line, move to the third line and delete the fourth line.

$ sed '2~3d' file.txt

1, +3d

The below given sed example will delete the first line, then go on to delete the next third line.

$ sed '1, +3d' file.txt

The below example will delete the second line, then continue to delete next four lines.

$ sed '2, +4d' file.txt

1,3!d

This will delete all lines excluding line numbers from 1 to 3. Here, '!' instruct sed not to delete the line number which conjoins with '!'.

$ sed '1,3!d' file.txt

Similarly, to delete all lines excluding line numbers 5 to 8, following sed command can be used.

$ sed '5,8!d' file.txt

Sed with I

'I' Ignore case is used to make the search pattern case insensitive.

sed 's/search_pattern/replace_pattern/I <file name>
# sed 's/abc/xyz/I' file1.txt

The above command with I will not bother about if it's ABC, aBc or AbC. I command completely makes your pattern search insensitive. Let's see some more examples.

# sed -n '/line/I p' file1.txt

You can even modify your search pattern in the way it's given below but with I command sed will match the pattern.

# sed -n '/LiNe/I p' file1.txt

Sed with e

The 'e' option with sed command is used to execute multiple sed commands. You can combine multiple commands with option e. You have to use '-e' before each command. Look at the below example.

# sed -e 's/abcd/dcba/' -e 's/qrst/tsrq/' file1.txt

The long argument of e is '--expression='

# sed --expression='s/abcd/dcba/' --expression='s/qrst/tsrq/' file1.txt

Hence, multiple -e command can be used to build a script. The below script would first replace and then delete lines containing the given pattern.

# sed -e 's/dcba/dcba/' -e '/tsrq/d' file1.txt

If you want you can combine multiple sed commands in an alternative way by using shell ($) to insert new line.

sed 's/abcd/dcba/'$'\n' 's/qrst/tsrq/' file1.txt

Sed with = sign

"=" command is used to print line number of the searched pattern in sed. There are various tools for finding line number of the searched string. I will explain about each of these tolls and their uses.

Suppose there is a file which contains the word 'sed command in unix' in it. Now, We will print the line number which contains the searched pattern.

With sed you can do it with the below command.

sed -n '/sed command in unix/ =' file.txt

Output:
1

The file contains the word in line 1. So, it's printed the line number as 1.
The 'n' option suppress the printing of pattern space.

With grep tool you can do the following to print line number.

cat file.txt | grep -n 'sed command in unix'

With awk use the below command.

awk '/sed command in unix/{print NR}' file.txt

With perl you can try the following command.

perl -ne 'print $.,":",$_ if /sed command in unix/' file.txt

I think you have noticed above that using sed is more simpler in doing the same task than other tools.

Sed n option for not printing

The sed n option is used for not printing anything. It suppress automatic printing of pattern. It can act like a grep command if we use n option with sed. The below sed command will search for the pattern and print it if found.

# sed -n '/pattern/p' file.txt

Sed generally processes each lines and prints lines which we tell it to get printed. '-n' disables the automatic line printing which you don't tell specifically to get printed.

Let's print lines from files.

# sed '1, 2p' file1.txt
Line 1
Line 1
Line 2
Line 2
Line 3
Line 4

Now, use '-n' to disable automatic printing.

# sed -n '1, 2p' file1.txt
Line 1
Line 2

This sed program replaces pattern 'word' to 'door' but won't print anything because of 'n' disabling printing anything.

# sed -n 's/word/door/' file1.txt

This prints only line where the pattern replacement happens.

# sed -n 's/word/door/p' file1.txt

Basic Sed command in UNIX and Linux to replace string

Sed ( Stream Editor ) is a command which is used for replacing and modifying text in a file. Although Sed is used to search and replace string, but you can also use sed command for other tasks.

Note: Sometime you ought to be very cautious while using sed to replace any string. For example the following command will replace all the string 'he' with string 'she'. Suppose if the file.txt contains a sentence "He board the aeroplane today morning to reach New York". The output of the below command would be.

"He board tshe aeroplane today morning to reach New York". Notice the word "the" becomes "tshe", beacause 'the' also contains the word 'he' in it.

$ sed 's/he/she/' file.txt

Sed replace string

Sed replace or sed substitution both are same. Sed is mostly used for substitution task. I have seen most of the people using sed just for substitution. 
Let's first create a file called 'sedfile.txt' and edit the file as given.

# cat > sedfile.txt
I am using Windows since I was in college
linux is the best open source OS

The most common use of sed is to replace any string in a file. This is an example where I am replacing the string "Windows" with "Linux". We can run the below command  to check before actually make permanent modification to the file.

# sed 's/Windows/Linux/' sedfile.txt

I am using Linux since I was in college
linux is the best open source OS

There is one more way of making the permanent change in the file. You can print the modified output to another file. Use the below sed command as given. Now at this time it will just copy the modified file to another file called “sedfileModify.txt”

# sed 's/Windows/Linux/' sedfile.txt > sedfileModify.txt

In the above example although sed replaces the string with a new one but it can only replace the first occurrences in a line. 

sed 's/Windows/Linux/' sedfile.txt
Output:
I am using Linux since I was in college. Windows is my favorite.
linux is the best open source OS

Notice the sed command able to replace only the first word 'Windows' to 'Linux' in the line.


Sed examples to replace nth pattern in a line

In the above examples sed command replaces only the first matching pattern in a line. To replace the nth pattern you have to execute the sed command in a different way.
For example I have edited the file 'sedfile.txt' as given below.

# vi sedfile.txt

I am using Windows since I was in college. If I have to choose I will choose Windows
linux is the best open source OS
~
~

To change only the 2th pattern execute the command as given below.

# sed 's/Windows/Linux/2' sedfile.txt

Similarly, you can replace any number of pattern in a file. To replace the third pattern in a line you can use '3'

 

Replacing all the pattern in a line

By default sed replaces only the first occurrence of the pattern. To replace all the pattern in a line you have to  use the global option 'g'. The following sed command will replace all the "Windows" string in the file to "Linux".

# sed 's/Windows/Linux/g' sedfile.txt


Replacing all the pattern from the nth pattern

The below command will replace all the matching pattern from the 2th pattern. It leaves the first match it found and replaces the rest of the match with the given string.
I have edited the sedfile.txt once more to be more comprehensive.

# vi sedfile.txt
I am using Windows since I was in college. If I have to choose I will choose Windows. Windows Windows
linux is the best open source OS
~
~
# sed 's/Windows/Linux/2g' sedfile.txt

I am using Windows since I was in college. If I have to choose I will choose Linux. Linux Linux
linux is the best open source OS


Replace string in a particular line

Well you may want to replace string only on a specific line, the following sed command can be used for that purpose. The given sed command will replace all the string 'MS' to 'OS' in fourth line.

$ sed '4s/MS/OS/g' file.txt

Similarly, you can use the same pattern to replace string from second, third or any given lines.

$ sed '2s/MS/OS/g' sedfile.txt


Replace string range of lines

Likewise, to replace string in a range of lines from 3 to 5, use the below sed command.

$ sed '3,5s/OS/MS/g' file.txt

Now, replace string in a range of lines from 2 to 4.

$ sed '2,4s/OS/MS/g' file.txt

Using other delimiter instead of slash (/) delimiter

We could use any delimiter in sed want but more conveniently we use slash (/) with our sed commands. Using too many slash delimiter at a time makes the Sed command looks messy if your command is little lengthy. For example you may want to change http:// to www. The http:// url already contains slash delimiter, using slash (/) again with the command will definitely make the command looks very confusing. In that case using other delimiters instead of slash would be more tedious.

Other delimiters you can use with your command are under score (_), (/), pipe (|), colon (:), comma (,), etc.

The following command will throw an error if you use front slash delimiter (/) as given below.

# sed 's/http:///www/' url.txt 
 sed: -e expression #1, char 10: unknown option to `s'


You can alter the same command in the following way to change http:// to www.

# sed 's\http://\www\' url.txt
www

or

# sed 's_http://_www_' url.txt

or

# sed 's|http://|www|' url.txt

Let's see how to use other delimiters I have mentioned above.

Consider you have a long file path given in your file. For instance.

/proc/fs/dec/scsi  and you want to change it to this /proc/sys/dev/scsi. The expressions are.

using colon (:)

sed 's:/proc/fs/dec/scsi:/proc/sys/dev/scsi:' file1.txt

using under score (_)

sed 's_/proc/fs/dec/scsi_/proc/sys/dev/scsi_' file1.txt

using pipe (|)

sed 's|/proc/fs/dec/scsi|/proc/sys/dev/scsi|' file1.txt

using comma (,)

You can even use comma if you want. I myself use comma sometimes but I don't prefer it.

sed 's,/proc/fs/dec/scsi,/proc/sys/dev/scsi,' file1.txt

 

Advance Sed command in UNIX and Linux to search and replace string in Linux and Unix


Adding extra character

Some times you may just want to add an extra character to the existing pattern. You can do that by '&' with the sed command.

Lets see some simple examples given below.

Example:1

# sed 's/linux/[&]/' sedfile.txt
Eat [linux] drink linux sleep linux
I am using Windows since I was in college.If I have to choose I will choose Windows. Windows Windows
[linux] is the best open source OS

Example:2


Adding '&&' with the sed command copying the first searched pattern 'linux' and modfying it to l'inuxlinux'

# sed 's/linux/[&&]/' sedfile.txt
Eat [linuxlinux] drink linux sleep linux
I am using Windows since I was in college.If I have to choose I will choose Windows. Windows Windows
[linuxlinux] is the best open source OS

Example:3

# sed 's/linux/& and Unix/' sedfile.txt
Eat linux and Unix drink linux sleep linux
I am using Windows since I was in college.If I have to choose I will choose Windows. Windows Windows
linux and Unix is the best open source OS


To change the position of character with sed

Some times you may also like to change the position of characters in a line. I will show you how to do it by the following example. As I explained it before you can use any delimiter you want according to your convenience. I have used here delimiter ( | ).

# sed 's|^\(.\)\(.\)\(.\)\(.\)|\4\3\2\1|' sedfileModify.txt 

taElinux drink linux sleep linux
ma I using Linux since I was in college
unilx is the best open source OS


Making changes to the string with \1, \2, \3

You can make changes to the string. You can replace the string with its double.
In the below example single 'linux' is being replaced with double 'linuxlinux'

# sed 's|\(linux\)|\1\1|' sedfileModify.txt 

Eat linuxlinux drink linux sleep linux
I am using Linux since I was in college
linuxlinux is the best open source OS

In the below example the string 'linux' is being replaced with triple 'linuxlinuxlinux'

# sed 's|\(linux\)|\1\1\1|' sedfileModify.txt 

Eat linuxlinuxlinux drink linux sleep linux
I am using Linux since I was in college
linuxlinuxlinux is the best open source OS

The following is an another example of how to change the position of word with sed command.
We will be changing the string 'linuxunix' to 'unixlinux'

# cat file4.txt 

linuxunix is the best open source
linuxunix is the best open source

# sed 's/\(linux\)\(unix\)/\2\1/' file4.txt 

unixlinux is the best open source
unixlinux is the best open source


Duplicating a line

You can also duplicate the line where you replaced the string with 'p' flag.

# cat file5.txt 
How to duplicate a line in unixlinux with sed 

 

# sed 's/unix/linux/p' file5.txt 

How to duplicate a line in linuxlinux with sed
How to duplicate a line in linuxlinux with sed

If you don't wish to duplicate a line and only want to print the replaced line use the '-n' option with sed command. See the example given below.

# sed -n 's/unix/linux/p' file5.txt


How to replace string on a particular line

To replace a string on a particular line you can assign the line number to where you want to replace. Lets see by an example.

# sed '2 s/unix/linux/' file4.txt 

linuxunix is the best open source
linuxlinux is the best open source


Some times you may want to replace in range of lines. The syntax to replace string in range of line is.

# sed '1,4 s/unix/linux/' file4.txt 

linuxlinux is the best open source
linuxlinux is the best open source
linuxlinux is the best open source
linuxlinux is the best open source
linuxunix is the best open source
linuxunix is the best open source


Using $ with the sed command

$ option is very useful when you want to replace the string from particular line to the last line. $ sign indicates the last line where the string will be replaced. See the example given below where the string is being replaced from the third line to last line.

# sed '3,$ s/unix/linux/' file4.txt 

linuxunix is the best open source
linuxunix is the best open source
linuxlinux is the best open source
linuxlinux is the best open source
linuxlinux is the best open source
linuxlinux is the best open source


How to delete line by sed

There is also an option of deleting lines with sed by using the option 'd'.

This below command will delete the third line from the file.

Example:1

# sed '3 d' file4.txt

Example:2

The below command will delete range of lines. From line number 3 to the last line will be deleted.

# sed '3,$ d' file4.txt


You can also use sed command in unix to delete lines from files by other methods. Look at the example given.

# cat file1.txt 
 I have been using Windows since I was in college. 
 Windows is my favourite


To delete all the lines from 'file1.txt' use the below command as given.

# cat file1.txt | sed 'd'

Similarly to delete the particular line you can use '1d' to delete first line or '2d' to delete second line etc. with sed.

# cat file1.txt | sed '1d'
 Windows is my favourite

To search any string in a line you can use the sed command in unix and linux in a following way.

# sed -n '/linux/p' file4.txt

To check any matching lines from 'passwd' file in Linux or UNIX you can do the following.

# sed -n '/root/p' /etc/passwd

The above command will search for the line which contains the word 'root'. Infact it's very useful if you searching any user in the passwd file.

Sed to print duplicate lines

If you want to print a duplicate line for each line you can use the option 'p'.

# sed 'p' file4.txt

By default sed print duplicate lines if you tell it to print. This prints copy of lines which contains the matched pattern.

# sed '/pattern/p' file.txt


How to run multiple sed command in unix

Some times you may also want to run multiple sed command in unix and linux. In that case you may use the option 'e'.

# sed -e 's/linux/unix/' -e 's/college/school/' sedfileModify.txt
Eat unix drink linux sleep linux
I am using Linux since I was in school
unix is the best open source OS


Sed to print line which doesn't match

By using option !p , sed gives you the power of excluding lines matched with the pattern. Let's get into a bit detail here.

The given sed example will print all the lines which matches pattern 'board'.

$ sed -n '/board/p' testfile.txt

Now to exclude all the lines which matches pattern 'board", use '!p' option.

$ sed -n '/board/!p' testfile.txt

 

Replacing a line with a new line by sed

How about replacing a whole line with a new line. You can replace the whole line with the option 'c'.

Example:

# cat file6.txt

Windows is my favourite OS

After executing the below command the below change has occurred. The command will first search and match with the string 'Windows' then it will replace the whole line with a new line.

# sed '/Windows/ c "Linux and Unix is my favourite OS"' file6.txt
"Linux and Unix is my favourite OS"


Sed command in UNIX to add new line

With sed you can add new line when the match is found. You can add line either after the match or before the match

# cat filenewLine.txt
This is an example of sed in linux for adding new line

The option 'i' is used to add line before match is found.

# sed '/linux/ i Sed to add line before match' filenewLine.txt
Sed to add line before match
This is an example of sed in linux for adding new line

Similarly option 'a' can be used to add line after the match is found.

# sed '/linux/ a Sed to add line after match' filenewLine.txt
This is an example of sed in linux for adding new line
Sed to add line after match


Transforming case of letters

With 'y' option you can tranform the case of letter. You can change from upper case to lower case or lower to upper case.

# cat fileTransform.txt 

transforming into uppercase by sed command.

# sed 'y/t/T/' fileTransform.txt 

Transforming inTo uppercase by sed command.


Removing blank lines

You can remove blank lines within a file with the option 'd'.

# cat fileTransform.txt 

Sed command used to remove blank lines.

This is a new line


With the below command the blank space between the two lines has been removed.

# sed '/^ *$/d' fileTransform.txt 

Sed command to remove blank lines.
This is a new line .

The same can also be written as

$ sed '/^$/d' file.txt

Similarly, you can remove more than one spaces and replace them with a single space with this simple sed command.

$ sed 's/  */ /g' filename


Sed script

You can also make a script file for sed to run a command. You can put the command into a script and run it as follows.
The following is a script to replace the string “Windows” to “Unix”

# vi script.sed
#!/bin/sed -f
s/Windows/Unix/g
~
~
~


You have to run the script as given

# sed -f script.sed file1.txt 

I have been using Unix since I was in college.
Unix is my favourite 


How to replace a line starting with a space into a new line

Sed is very handy if you want to remove the new line which starts with a space, and add it to the first line.

Create a script as follows.

# vi replace_script.sed 

#!/bin/sed -f
N
s/\n / / 


The file5.txt file has some lines which have unnecessary spaces before the line starts.

# cat file5.txt 

Linux is my favourie,
I love to work on linux.
Sed is a command
which can be used for replacing string.
Sed is also
used in Unix.

Below is the command when executes removes the spaces and adding the line to the first line.

# sed -f replace_script.sed file5.txt 

Linux is my favourie, I love to work on linux.
Sed is a command which can be used for replacing string.
Sed is also used in Unix.

 

Sed regex examples

The given sed regex examples will make you clear about some special characters which can be used with sed command in UNIX or Linux for regular expressions. We will also learn about how to use each of the given characters with sed command.

. PERIOD
* ASTERIS
^ CARET
$ DOLLAR


Period

It matches any single character between given characters. For example /c.d/ matches can be cad, c-d, c+d, c1d, c4d ,etc.
The file 'file.txt' contains the following lines.

$ cat file.txt
bcd
dca
rdp
cad
c-d
c+d
c1d
c4d

Now using the dot(.) with sed command gives the following output. It matches any single character between /c.d/ and prints on the terminal.

$ sed -n '/c.d/p' file.txt 
cad
c-d
c+d
c1d
c4d

Asterisk

This will match any ocuurance of the character. For example /c*d/ will match cad, dca, acd or cfdgh etc. The below sed example will given the following output of of the file 'file.txt' on the terminal.

$ sed -n '/c*d/p' file.txt 
bcad
dcf
abced
tyyctd
cfdrry


Caret

It will match the character in the beginning of the line in a file. For example, to print all the lines having 'c' at the beginning of the line, use the below sed command.

$ sed -n '/^c/p' filename

Dollar

Dollar ($) sign is used to match the end of line. The given sed example will print all the lines which ends with 'd'.

$ sed -n '/d$/p' file.txt 
bcad
abced
tyyctd

Sed --version

With --version option you can find out current version of sed.

# sed --version

Sed -h Help

The option -h or --help with sed command in unix will show you the summary of the options that you can use with sed.

# sed -h

or

# sed --help

 

Sed hold buffer and pattern buffer

Sed has two buffers Hold buffer and Pattern buffer.

Sed Pattern buffer is like the temporary space where the current information is kept. When we execute a sed command, it reads a file and store the read information in the pattern buffer. We get the printed output on the terminal from the information kept at pattern buffer.

Sed Hold buffer is a long storage memory where the information is stored for a longer period of time. The information kept at hold buffer can be retrieved subsequently. Informations are generally copied from the pattern space to hold space.
Tasks are not performed directly on hold space rather some functions are there to execute the task. These functions are.

1.Hold function (H, h)
2.Get function (G, g)
3.Exchange function (x)

H function

Sed H function appends contents from pattern space to hold space.

h function

Sed h function copies contents from pattern space to hold space.

G function

The sed G function appends contents from hold space to pattern space.

g function

Sed g function copies contents from hold space to pattern space. In this action previous contents gets deleted.

x function

The sed x function (exchange function) as the name suggests exchange spaces between hold buffer and pattern buffer.

How to join two lines

You can join two lines with sed command by using the following command. Let's join line 1 with 2 of the file.

# cat file.txt

sed is a c
ommand for stream editing
# sed '1{N;s/\n//}' file.txt

sed is a command for stream editing


Man sed (Man page of sed command)

In this section we will discuss about some flags or options given in the man page of sed command in UNIX and Linux.

Sed f example

With the 'f' option we can specify the contents of the file to be executed as a process. Lets see an example to make it more clear.

Create a file 'filetest.txt' and save it as given.

$ vi filetest.txt
/DT/{s//'"$(date +%F,%T)"'/;s/.*/echo '&'/e}

The following sed command will execute the script given in the 'file testfile.txt'. Notice the option 'f' is used here.

$ echo "Now the Date and Time is DT" | sed -f testfile.txt
Now the Date and Time is 2016-10-17,02:48:06

Suppose there is a symlink file and if you do not use option --follow-symlinks, then sed will destroy the link file and create a regular file in place of link file.So, be very careful when editing any symbolic link file.

The below sed command example will find the match and replace the word in a symlink file without destroying the link.

$ sed --follow-symlinks 's/match/replace/g' filename.txt

When you become sure about the outcoming of the command, then use option 'i' for in-place editing in the file.

$ sed --follow-symlinks -i 's/match/replace/g' filename.txt

Sed i option example

In many previous examples this option has been explained. So I will just summarize it here. It is used to in-place editing in the file. In the below example it will find the string 'match_word' and replace it with the string 'replace_word' in the 'myfile.txt' file.

$ sed -i 's/match_word/replace_word/g' myfile.txt

Sed binary mode (--binary) or -b

If you look into the man page of the sed command you will see that '--binary option is used for compatibity with WIN32/CYGWIN/MSDOS/EMX.
The -binary option will instruct the sed command to open file in binary mode.

If you are using CYGWIN sed in Windows, try to use the following sed substitution example with a file. It will convert the ending of the line feed to UNIX(\n).

$ sed 's/any_string/replace/ filename

Thus, use the below sed command to open input file in binary mode.

$ sed -b 's/any_string/replace/ filename

Sed line wrap (-l)

It states desired line-wrap for the list 'l' command. Late me make it more clear by an example.
The contents of a file myfile.txt is.

$ cat myfile.txt
LINUX UNIX SED GENTOO REDHAT UBUNTU CENTOS

Let's list the content of the file by sed command with list 'l' option.

$ sed 'l' myfile.txt
$
LINUX UNIX SED GENTOO REDHAT UBUNTU CENTOS$
LINUX UNIX SED GENTOO REDHAT UBUNTU CENTOS

It is also printing the duplicate content. Suppress it by '-n' option and lets use the command again.

$ sed -n 'l' myfile.txt
$
LINUX UNIX SED GENTOO REDHAT UBUNTU CENTOS$

Did you see the output, the duplicate line is supressed here. Now we will line-wrap 5 length character as given below.

$ sed -n -l 5 'l' myfile.txt
$
LINU\
X UN\
IX S\
ED G\
ENTO\
O RE\
DHAT\
 UBU\
NTU \
CENT\
OS$

Sed unbuffered (-u)

When sed is not printing anything on terminal that means it is working in buffered mode. By default sed first packs its buffer before processing on terminal, it increases the amount of data processing.If you want you can instruct the sed command to work in unbuffered mode with the sed option '-u'. The syntax of using the 'u' option with sed command is given below. You won't see any difference when processing small number of data with this command.

$ sed -u 's\find\replace\g' filename

Sed --separate (-s)

The '-s' option or 'separate' allows the sed command to treat each files differently when executing. For example I want to print line number 2 of the two files. So, this is how we will execute the command.

$ sed -n '2p' sed_file myfile

But wait a minute there is a problem in the above command. It will print only the line number 2 of the first file. But why is it so?
Because by default sed concatenate all the files. In this case sed prints line number 2 of the file after concatenation.
Therefore, to treat each file differently, we need to use a separator that is '-s' option with sed.
Now execute the following on your terminal. It will print line number 2 of both the files.

$ sed -n -s '2p' sed_file myfile

Sed with c\

With the option 'c\' you can delete lines of matching word. For example the below sed example will delete the line matching with the pattern.

$ sed '/matched_pattern/c\' myfile

Instead of using 'c\' to delete a line I would rather advice you to use 'd' with sed to delete lines. I have already described it of how to use 'd' option with sed in the previous section of this article.

Change line with sed 'c'

With sed c you can even change the current line in a file with a new line.
Let's replace line third in the file.

$ cat file.txt
Line 1
Line 2
Line 3
Line 4
$ sed '/3/ c\We will replace line third' file.txt
Output:
Line 1
Line 2
We will replace line third
Line 4

Add the option '-i' to make the change permanent.

$ sed -i '/3/ c\We will replace line third' file1.txt

With sed c you can even add multiple new lines at a time. Let's create a sed command for this.

sed '
/4/ {
i\
This is line fourth
a\
This is line sixth
c\
This is line fifth
}' file.txt
Output:
Line 1
Line 2
Line 3
This is sed command in unix to add line fourth
This is sed command in unix to add line fifth
This is sed command in unix to add line sixth

'i\' inserts a new line line here
'a\' appends a new line after the matched pattern.

Sed append (a\)

This option is used to append a line after a matching word. The syntax is

$ sed '/match_word/a\Add this line' myfile.txt

So the above sed command will append a new line "Add this line" after the matched word.

Sed q command for exit

The sed option q is used to quit after certain task is reached. You can use it to read lines but you cannot use it to read range of lines.

$ sed '4 q' myfile

It will read lines from 1 to 4 then quit.
if you use the command as given below, it will throw error.

$ sed '2,3 q' myfile

sed: -e expression #1, char 5: command only uses one address

Sed w flag

You can write your changes to another file on runtime with 'w'.

$ sed 's\this\that\w change' <myfile.txt

The above command is a simple search and replace sed command example. In the example 'this' is being replaced by 'that'. At the same time a file called 'change' will be created where your modified data will be saved.

Using flag y

In sed you can change letters from uppercase to lowercase or lowercase to uppercase with the option 'y'. You can either choose to covert all letters or you can choose some specific letters.  See the example below.

$ sed 'y/ABCD/abcd/' filename

It will substitute (uppercase to lowercase) all the character it finds from A through D.

Similarly, you can convert letter from lower case to upper case. with the below sed command.

sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' filename

This substitute all small letters to capital letters in a file.

The below sed command perhaps is less frequently used, though it's useful to know about it.

$ echo command for sed | sed 'y/abcd/1234/'
Output:
3omm1n4 for se4

Sed branching (Labels)

Although sed is a stream editor, it is also a compact programming language and uses many of the programming features. One of the feature of a programming language is flow control. You can change the flow control in sed  by defining label.  A label is stated with a text preceded by a colon. Let's see how to change flow control.

There are two levels or you can call branching used by sed to control the flow of the sed command.
1. Unconditional branching
2. Conditional branching


b= b command is used for unconditional branching in sed. The b command is used to jump to a level where level is designated as :anylabelname

t= t command is used for conditional branching in sed. It is vey similar to b. However, in this case jumping to level with 't' occur only if the execution is completed fully till the last statement in the file.


 Command b
'b' as said before, it is used for unconditional branching. It works similar to 'go to' statement in other programming languages. Lets see an easy example to understand the concept clearly.
Create a file called 'label.txt'.

$ vi label.txt
Statement to test sed command
learn and understand the examples
this will test branching label b

Now run this command and see the output.

$ sed ':a;N;$!ba;s/\n/, /g' label.txt

Statement to test sed command, learn and understand the examples, this will test branching label b,

Lets see what's happening here in this command!!. The example has been taken from one of the stackoverflow example. I found it very helpful.

First-- Declared a level ':a'.
Second-- 'N' it reads file line by line and append next line to pattern buffer(pattern space). It is a temporary buffer or space where the current information is stored.
Third-- '$!ba' Looping back to level 'a' unless reached to the end of the file.
Fourth-- 's/\n/, /g'  command of search and replace. replaces every newline in the file with comma (,).

 Command t
The option 't' is used with sed for conditional branching. When you want to execute a level when a condition is fulfilled in a statement, then you have to use 't' with sed. For example see the below command where level 't' is used to jump to label when condition is fulfilled. It is a simple example for putting hash tag in the beginning of the line where pattern 'label' is found.

$ sed -n ':a;/label/s/^/#/; /##/!t a; p' level.txt
Statement to test sed command
learn and understand the examples
##this will test branching label t

In the above example the command will not execute completely until the hash tag count increases to two.
'^' means beginning of the line.


Learn How to import csv to mysql in Linux/UNIX


Tricks of sed command in UNIX and Linux

Following are some sed command examples which will help you learn some more cool tricks of sed.

Numbering each line of your file

You may want to number each lines in your file. In that case use the below sed command.

$ sed = myfile.txt | sed 'N;s/\n/\t/'
1    Sed
2    Linux
3    UNIX
4    AWK

Counting lines

Below sed example will print the total number of lines present in your files.

$ sed -n '$=' myfile.txt
4

Print lines having 20 or more than 20 characters in a file

If you want to print lines from your file having more than certain characters, then use the below sed command example.

$ sed -n '/^.\{20\}/p' myfile.txt


Delete all blank spaces from a file

The below sed examples can be used to delete all blank spaces from your file.

$ sed '/^$/d' myfile.txt


Print only valid email address from file

If you have files having number of email addresses in them, then the below sed command is very useful to extract valid email addresses from the file.

$ sed -nr '/^[^@]+@[^.]+\.com\s*$/p' myfile.txt

 

Using alphabet character class keyword

There are some important alphabet character class keyword. Their usage and how to use them with sed are as follows.

[[:alpha:]] - Print lines having alphabet
[[:digit:]] - Print lines having digit
[[:alnum:]] - Print lines having alphanumeric
[[:upper:]] - Print lines having uppercase
[[:lower:]] - Print lines having lowercase
[[:punct:]] - Print lines having punctuations

The below UNIX sed command will print the line having atleast single alphabetic character.

$ sed '/[[:alpha:]]/p' filename

The given sed example will print lines having atleast single digit.

$ sed '/[[:digit:]]/p' filename

For printing lines which contains punctuation, use the given sed example.

$ sed -n '/[[:punct:]]/p' file.txt
windows?
linux;
456hgy.
mac,
ghy458!


With the use of caret (^) you can make the sed command more powerful. Say for instance , you want to print those lines which are starting with digit, then you need to use the command as follows.

$ sed '/^[[:digit:]]/p' file.txt
123abcd

 

Sed & (Ampersand) trick

Sed & is very special. It means the whole part of the pattern matched. Look at the few examples given below to understand the use of '&' with sed.

Say, you have a file which contains different phone numbers as given below.

$ cat file.txt
555354689
245356879
124556824

Now, if you want to insert hypen after every second digit in the phone number. So, you code would be as given below.

$ sed 's/^[[:digit:]][[:digit:]]/&-/g' file.txt 
55-5354689
24-5356879
12-4556824

In the above, one [[:digit:]] represents single digit. '&' here represents the matched digit. So, we are inserting '-' after the matched digit.

You can also write the above code as follows. '\{2\}' is the total number of digits.

$ sed 's/^[[:digit:]]\{2\}/&-/g' file.txt

 


This is it with Sed command in UNIX and Linux examples tutorial. Please give your valuable suggestions.

Also See:
Rsync exclude directory (folder), files in Linux with examples
Open source mail server - set up Rainloop open source mail server


References:
Wikipedia
Sourceforge


That's it with the tutorial Sed command in UNIX and Linux examples  - How to use. Don't forget to share

Download Our Free eBook now

Linux and UNIX Shell scripting ebook

7 Comments on Sed command in unix and Linux examples – How to use

Leave a Reply

Your email address will not be published.


*


shares