قالب وردپرس درنا توس
Home / Tips and Tricks / Using Pipes on Linux

Using Pipes on Linux

  Linux Terminal on a Laptop
Fatmawati Achmad Zaenuri / Shutterstock.com

Use Linux pipes to choreograph the collaboration of command-line utilities. Simplify complex processes and increase your productivity by using a series of stand-alone commands and transforming them into a stand-alone team. We show you how.

Pipes Are Everywhere

Pipes are one of the most useful command-line features available on Linux and Unix-like operating systems. Pipes are used in countless ways. Look at a Linux command-line article ̵

1; on every website, not just ours – and you'll find that pipes are more common. I've talked about some How-To Geek Linux articles, and pipes are used in one way or another in all.

With Linux pipes, you can take actions that the shell does not support by default. But since the Linux design philosophy is to have many small utilities that perform their dedicated functions very well and without unnecessary functionality – the mantra "make one and do it well" – you can tailor command sequences along with pipes so that the Issue one command will be entering another. Every command you enter brings its unique talent to the team, and you've soon put together a successful squad.

A Simple Example

Let's say we have a directory with many different file types. We want to know how many files of a certain type are in this directory. There are other ways to do this, but the goal of this exercise is to introduce pipes. So we'll do it with pipes.

With ls we can easily retrieve a list of files.


  Collection of Files in a Directory in a Terminal Window

To distinguish the desired file type, we use grep . We would like to search for files whose file name or file extension contains the word "page".

We use the shell special character " | " to direct the issue of ls . in grep .

  ls | grep "page" 

  ls -l | grep

grep prints lines that match the search pattern. This gives us a listing that contains only ".page" files.

 Listing of paging files in a terminal window

Again, this simple example shows the functionality of pipes. The output of ls was not sent to the terminal window. It was sent to grep as data for the command grep to work with. The output we see comes from grep the last command in this chain.

Extension of our chain

Let's start with the extension of our chain of pipe commands. We can count the ".page" files by adding the command wc . We will use the option -l (number of lines) with wc . Note that we have also added the option -l (long format) to ls . We will use this soon.

  ls - | grep "page" | wc -l 

  ls - | grep

grep is no longer the last command in the chain, so we do not see its output. The output of grep is fed into the command wc . The output in the terminal window is from wc . wc reports that there are 69 ".page" files in the directory.

Let's expand things again. We take the command wc from the command line and replace it with awk . The output of ls with the option -l (long format) contains nine columns. We will use awk to print columns five, three, and nine. These are the size, the owner and the name of the file.

  ls -l | grep "page" | awk # {print $ 5 "" $ 3 "" $ 9} & # 39; 

  ls -l | grep

We get a list of these columns for each of the matching files.

 Three columns for each matching file in a terminal window.

We now hand over this issue via the command sort . We will use the option -n (numeric) to tell sort that the first column should be treated as numbers.

  ls -l | grep "page" | awk # {print $ 5 "" $ 3 "" $ 9} & # 39; | sort -n 

  ls -l | grep

The output is now sorted in the order of file size, with a custom selection of three columns.

 Files sorted by size in a terminal window

Adding another command [19659005] Finally, we add the command tail . We recommend listing only the last five output lines.

  ls -l | grep "page" | awk # {print $ 5 "" $ 3 "" $ 9} & # 39; | sort -n | Tail -5 

  ls -l | grep

This means that our command is something like "show me the top five" .page "files in this directory, sorted by size". Of course, there is no order to accomplish this, but with the help of Pipes, we have created our own. We could add this or another long command as an alias or shell function to store all the input.

Here is the output:

 Five largest .page files sorted by size in a terminal window

We could reverse the order of magnitude by using the option -r (Reverse) to the command sort and head instead of Tail to select the lines above in the output.

   ls -l | grep 

This time, the top five largest ".page" files are listed from largest to smallest:

 Five largest .page files in inverse order of size in a terminal window

Some recent examples [19659005] Here are two interesting examples from current quick guides.

Some commands, such as the command xargs have been developed for forwarding input. Here's a way how we can count wc the words, characters, and lines in multiple files by passing ls into xargs then the list to feed the filename wc as if it had been passed to wc as a command line parameter.

  ls * .page | xargs wc 

  ls * .page | xargs wc in a terminal window

The total number of words, characters, and lines is listed at the bottom of the terminal window.

 Number of words, characters, and lines in a terminal window

This allows you to retrieve a sorted list of unique file extensions in the current directory with a number of individual types.

  ls | rev | cut -d & # 39;. & # 39; -f1 | rev | sort | uniq -c 

  ls | rev | cut -d & # 39;. & # 39; -f1 | rev | sort | uniq -c in a terminal window

There's a lot going on here.

The output shows the list of file extensions, sorted alphabetically by the number of unique unique types.

 List of unique file extensions in a terminal window

Named Pipes

There is another type of pipe called Named Pipes. The pipes in the previous examples are immediately created by the shell when it processes the command line. The pipes are created, used and then discarded. They are transient and leave no trace of themselves. They only exist as long as the command that uses them is executed.

Named pipes appear as persistent objects in the file system so you can view them with ls . They are persistent because they survive a computer restart even though all unread data is discarded at that time.

Named pipes were often used concurrently to allow different processes to send and receive data, but I have not seen them that way for a long time. No doubt there are people who still use them with great effect, but I have not met anyone lately. For the sake of completeness or just to satisfy your curiosity, you can use it here.

Named pipes are created with the command mkfifo . This command creates a named pipe named "geek pipe" in the current directory.

  mkfifo geek-pipe 

  mkfifo geek-pipe in a terminal window

The details of the named pipe are displayed pipe, if we use the command ls with the option - Use l (long format):

  ls -l geek pipe 

  ls -l geek pipe in a terminal window

The first character of the collection is a "p", which means that it is a pipe. If it were a "d", this would mean that the file system object is a directory, and a hyphen "-" would mean that it is a regular file.

Using the Named Pipe

Let's use our pipe. The unnamed pipes we used in our previous examples passed the data immediately from the send command to the receive command. Data sent via a named pipe remains in the pipe until it is read. The data is actually in memory, so the size of the named pipe in the collections of ls does not depend on whether there is data in it or not.

We will use two terminal windows for this purpose. I am using the label:

  # Terminal-1 

in a terminal window and

  # Terminal-2 

in the other so that you can distinguish between them. The hash character "#" tells the shell that the following is a comment and ignores it.

Take the whole previous example and redirect it to the named pipe. So we use both unnamed and named pipes in one command:

  ls | rev | cut -d & # 39;. & # 39; -f1 | rev | sort | uniq -c> outside pipe whistle 

 ls | rev | cut -d & # 39;. & # 39; -f1 | rev | sort | uniq -c> geek-pipe in a terminal window

Not much is going to happen. However, you may find that you are not returning to the command prompt, so something is going on.

In the other terminal window, type the following command:

  cat <geek-pipe 

<img class = "alignnone size-full wp-image-439186" data-pagespeed-lazy-src = "https: //www.howtogeek.com/wp-content/uploads/2019/09/x20.png.pagespeed.gp+jp+ jw + pj + ws + js + rj + rp + rw + ri + cp + md.ic.eHaOfO4Dp2 .png "alt =" cat < geek-pipe in a terminal window" width="646" height="77" src="/pagespeed_static/1.JiBnMqyl6S.gif" onload="pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);" onerror="this.onerror=null;pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon(this);"/>

We redirect the contents of the named pipe to cat so that cat displays this content in the second terminal window, here is the output :

 The contents of the named pipeline appear in a terminal window [19659006] And you will see that you returned to the command prompt in the first terminal window.

 Completed task and command prompt in the terminal window

So what just happened.

  • We redirected some issues to the named pipe.
  • The first terminal window returned not return to the command prompt.
  • The data stayed in the pipe until it was read from the pipe in the second terminal.
  • We returned to the prompt in the first terminal window. [19659092] You may think you could run the command in the first terminal window as a background task by appending & at the end of the command. And you are right. In that case, we would have returned to the prompt immediately.

    If the background processing was not used it should be emphasized that a named pipe is a blocking process . When you put something in a named pipe, only one end of the pipe is opened. The other end is not opened until the reader extracts the data. The kernel interrupts the process in the first terminal window until the data is read from the other end of the pipe.

    The Power of Pipes

    Named pipes are a kind of novelty today.

    Normal old Linux pipes, on the other hand, are one of the most useful tools you can have in your Terminal Window toolkit. The Linux command line comes to life for you and gives you a completely new startup if you can orchestrate a collection of commands to achieve a coherent performance.

    Farewell Tip: It's best to write your pipe commands by adding one command after another and running that part, then initiating the next command.

Source link