Using Command Line Tools to Aid Development – Part 2 (Sed)

The first part of this series of articles is here and I recommend you start with that one to get an idea of this series of articles and why using command line tools to aid your development (in any language/environment) can be very useful.

I am going to dive right in here with another text based tool called ‘sed’. Sed stands for ‘stream editor’ and it is another very early Unix days tool created in 1974, its goal being a streams based implementation of text editing that utilises regular expressions which back then was a new era for efficiently processing text.

These days GNU have their own offering of sed which has become the standard edition, this version includes some big improvements and new features including editing files in-place and other functionality for convenience. This package is default available on most Unix/Linux distros and can be installed for use on Windows based setups as well.

Once you have made sure it is available on your chosen development environment you can use sed for various operations that would take longer or be tedious if you were doing it manually or using GUI tools to accomplish. I will show examples of some of these situations for you below.

In modern programming there is a big emphasis on splitting your projects/scripts/programs into multiple neatly organised files that are in various formats and structures in order to keep your project to standards, make it easier for other developers to read & modify, for compiler rules etc. This can mean you end up needing to replace certain words, phrases, variable names and the like project-wide or folder-wide in order to update name changes and such. Sed can help you handle situations like this with ease.

Use – Replacing text in files
sed -i /<search>/<replace>/ <filename>
sed -i s/abc/cba/ file1
Use – Replacing text in all files in a folder/project
sed -i /<search>/<replace>/ *
sed -i s/abc/cba/ *
Use – Cleaning up code in your project or file

You can go further with this sort of concept and start using the functionality of sed to clean up your code, remove unwanted function calls and such with commands such as the ones below.

sed -i '/^ *$/d' <filename>
^ This removes all lines with just whitespace or that are blank.

sed -i s/<function name>\(\)\;// <filename>
^ This would remove all calls in a file or project to a certain function.

sed -i '/\s*#.*$/d' <filename>
^ This would remove all comments that use the # style from your file or project.

sed -i s/<old variable/function/class name>/<new variable/function/class name>/ <filename>

^ This last one is self explanatory really but it is very useful for when you need to change the name of something that gets referenced or called all over the place in a large project.

You will notice the different styles of sed use that you can do, single quoting can be useful if your using certain characters in your command that will upset your preferred command line shell. The function call example above shows the other method of escaping your special characters using \, this is just another way to do the same thing.

Advanced switches and uses

Once you get the hang of some basic sed commands you can use some of the switches to chain commands, store commands in sed ‘script’ files etc, here is some examples of switches:

The -e switch can be used for chaining 
sed -i -e '<command>' -e '<command>' <filename>
sed -i -e '/\s*#.*$/d' -e '/^ *$/d' <filename>
^ This chains the commands up to remove all # style comments and all blank/whitespace lines from your file or project, you can chain up more expressions and eventually come up with one-liners to clean your code up when required.

You can create sed scripts by just putting one sed command per line into a file and calling it with the -f switch.

File x.sed:

/^ *$/d


sed -f x.sed <filename>

Creating little sed scripts and keeping them handy to clean up code or even data-files (e.g. simple csv processing) can be a good use of this feature.

This concludes our introduction to using the great command line tool ‘sed’ for aiding development. There is lots of other great ways you can use sed to process any type of text file or even streams from pipes similar to grep in the first article to leverage the most out of its functionality. If you need a more adaptable advanced version of what sed does with even more functionality to really mangle your text based files definitely look into the command line tool AWK.

There will be a part 3 coming soon, I apologise for the long waits between my articles.

Live the dream! 🙂