In the world of development today everyone is looking to speed up the process while remaining efficient and not cutting corners, there is lots of different ways to achieve this but a lot of the time I find developers miss using some simple command line tools that can help certain things such as locating every use of a keyword (be it a variable name, function name, value etc) in an entire project and their locations to using linters to help you brush up to the strictest of standards.
Between the Bash shell, GNU/Linux and other command line tools you can save yourself time and pain while also boosting your overall skill set. This is going to be part one of a few parts where I will show not just the tools themselves but a couple of use cases that actually help during handling the development or maintenance of code/projects.
Introducing… grep, taken from Wikipedia the standard descriptive line would be ‘grep is a command-line utility for searching plain-text data sets for lines that match a regular expression’. As a base description that is pretty accurate but as time has gone on (the first version was released 1974 for the original Unix operating system) it has now got quite a few features and uses.
Use – Searching a file for occurrences:
The plainest use of grep would be to purely search a file for an occurrence of a string. You can specify a regular expression or just a search string like so:
grep '<search>' <filename> e.g. grep test file.txt grep 'test' file.txt grep ^test$ file.txt
Note: The use of quote marks around the search term is optional but should be used if search contains any characters that would trigger the Bash shell (or your preferred platform) to interpret them as a split of the command/line or things like pipes etc.
Use – Searching wider for occurrences:
You can use grep to search multiple files all the way up to multiple directories recursively file-by-file.
grep <search> <multiple files> e.g. grep test file1.txt file2.txt file3.h file4.cpp file5.txt grep -R <search> <directory> e.g. grep -R test test_files/ grep -R test test_directory1/ test_directory2/ grep -R test * - to search all files in current directory and all sub-directories, wildcard style.
You can also mix file & directories in the search radius.
grep -R test.txt test_files/
You will notice immediately when you give more than one file or any directories to be searched by grep that it will start reporting the file that the occurrence was found in at the start of each line so you can find which files contain what you were searching for.
livethedream$ grep -R test test.txt some_files/ test.txt:testing test.txt:test livethedream$
You can use grep with the wildcard solution in the root of your or other people’s code, libraries, projects etc to find where a function/class/variable is declared, defined, used etc this is really helpful in speeding up trying to find your way around code you either did a long time back or you are taking over maintaining from another developer or even modifying 3rd party code for your own devices.
Use – Searching the pipe for occurrences:
Grep would not be what it is today without its use when put together with the traditional stream pipes that operating systems have been making use of since around the time of its first version. In a traditional Bash shell setup on a Unix/Linux system the pipe symbol | works by ‘piping’ the output of the command on the left hand side to the command on the right hand side and this can be chained over and over again for great effect.
Here is a typical example of using grep with a pipe to search the output of any command you can execute that has output:
<command> | grep <search> e.g. livethedream$ ls file1.txt file2.txt file3.txt file4.txt file5.txt test.txt test_files livethedream$ ls | grep test test.txt test_files livethedream$
You can get really good results using grep in this way in regards to the development process by combining it with the output of tools like package managers, compilers, shell scripts, source control tools (git, hg, svn etc).
Another good use in the pipe instance is the -v switch which will reverse the usage to remove the occurrences rather than return them, so taking our above example lets try the same with -v.
<command> | grep <search> e.g. livethedream$ ls file1.txt file2.txt file3.txt file4.txt file5.txt test.txt test_files livethedream$ ls | grep -v test file1.txt file2.txt file3.txt file4.txt file5.txt livethedream$
You can see how this would be useful, you can even chain it instead of using a regular expression for quick removal of one or more separate searches like so:
livethedream$ ls | grep -v test | grep -v file1 | grep -v file3 file2.txt file4.txt file5.txt livethedream$
You can also use this method with commands like cat, curl etc to parse your way through various types of data removing different searches quickly without mucking about with regex – a good example just for thought is using cat to view a csv file and using grep -v with multiple pipes to remove all lines that have a certain field value, only leaving the data left.
A quick example of this applying for searching development projects could be wanting to search for a function, but only search for it when it is called in a certain way or without a certain argument. I add the ( after the function name as an example of narrowing down making sure it is a function called.
grep -R '<function name>(' * | grep -v '(true)'
This call would search for all the calls (and the definition, but that isn’t relevant) of a function inside the entire project/codebase but removes all the times it is called with just one argument with a value of true as a boolean.
This concludes the introduction to grep for use with searching through code specifically. There is a whole host of other applications for grep and there is a couple of other really useful switches that can change the regular expression engine in use (if you use the regex with it) or how it searches. Be aware grep has different versions depending on the operating system but mostly it is always the same in the way it acts for most basic operations.
In part 2 of this series I will go even further with helping the development process using command line tools by introducing another key program that has a long history starting from the days of original Unix.