[Unix-Linux] Pipes and Filters

📋 This is my note-taking from what I learned in the Unix/Linux Tutorial!

• Reference link: https://www.tutorialspoint.com/unix/index.htm

Pipes and Filters

Connect two commands together so that the output from on program becomes the input of the next program. Two or more commands connected in this way form a pipe.

• Pipe: Put a vertical bar (|) on the command line between two commands.
• Filter: When a program takes its input from another program, it performs some operation on that input, and writes the result to the standard output.

The grep Command

The grep command searches a file or files for lines that have a certain pattern.

$grep pattern file(s)

• grep comes from the ed (Unix line editor) command ‘g/re/p’ which means “globally search for a regular expression and print all lines containing it.”
• Regular expression → plain text(e.g. a word) and special characters used for pattern matching.

Look for a pattern consisting of a single word:

$ls -l | grep "Aug"
-rw-rw-rw-   1 john  doc     11008 Aug  6 14:10 ch02
-rw-rw-rw-   1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-r--   1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-r--   1 carol doc      1605 Aug 23 07:35 macros
$

Various options along with the grep command:

Sr.No	Option & Description
1	-v Prints all lines that do not match pattern
2	-n Prints that matched line and its line number
3	-l Prints only the names of files with matching lines (letter “l”)
4	-c Prints only the count of matching lines
5	-i Matches either upper or lowercase

Use a regular expression that tells grep to find lines with “carol”, followed by zero or other characters abbreviated in a regular expression as “.*”, then followed by “Aug”:

$ls -l | grep -i "carol.*aug"
-rw-rw-r--   1 carol doc      1605 Aug 23 07:35 macros
$

The sort Command

The sort command arranges lines of text alphabetically or numerically.

$sort food
Afghani Cuisine
Bangkok Wok
Big Apple Deli
Isle of Java

Mandalay
Sushi and Sashimi
Sweet Tooth
Tio Pepe's Peppers
$

Options that control the sorting:

Sr.No	Description
1	-n Sorts numerically (e.g. 10 will sort after 2), ignores blanks and tabs
2	-r Reverses the order of sort
3	-f Sorts upper and lowercase together
4	-k Specify which fields should be used for sorting

More than two commands may be linked up into a pipe.

List the files in long format, filter for files with a modified date in August using grep, and then sort them based on the numeric value of the sixth field using sort -k 6n:

$ls -l | grep "Aug" | sort -k 6n
-rw-rw-rw-  1 john  doc     11008 Aug  6 14:10 ch02
-rw-rw-rw-  1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-r--  1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-r--  1 carol doc      1605 Aug 23 07:35 macros
$

• ls -l: List the files in long format
• grep “Aug”: Filter for files with a modified date in August

• sort -k 6n: Sort them based on the numeric value of the sixth field

  • 6: Represent the field position. Sixth field is the month or day of the month

  • n: Specify the sorting types as numeric

The pg and more Commands

If we use more or pg command as a filter, the display stops once the screen is full of text.

$ls -l | grep "Aug" | sort -k6n | more
-rw-rw-r-- 1 john doc 8515 Aug 6 15:30 ch07
-rw-rw-r-- 1 john doc 14827 Aug 9 12:40 ch03
-rw-rw-r-- 1 john doc 16867 Aug 6 15:56 ch05
-rw-rw-r-- 1 carol doc 1605 Aug 23 07:35 macros
-rw-rw-r-- 1 john doc 2488 Aug 15 10:51 intro
...
--More--(50%)
$

• Press Enter to scroll to the next page or Q to quit.

---

Back to Top

See other articles in Category Unix-Linux