Pipes and filters

ls shows all files and directories at this point.

!ls

create-delete.ipynb           index.rst
create-delete.ipynb.license   pipes-filters.ipynb
file-system.ipynb             pipes-filters.ipynb.license
file-system.ipynb.license     shell-variables.ipynb
grep-find.ipynb               shell-variables.ipynb.license
grep-find.ipynb.license

With *.rst we restrict the results to all files with the suffix .rst:

!ls *.*

create-delete.ipynb           index.rst
create-delete.ipynb.license   pipes-filters.ipynb
file-system.ipynb             pipes-filters.ipynb.license
file-system.ipynb.license     shell-variables.ipynb
grep-find.ipynb               shell-variables.ipynb.license
grep-find.ipynb.license

We can also output only the number of lines, words and characters in these documents:

!wc *.*

     382     885    9242 create-delete.ipynb
       3       6      81 create-delete.ipynb.license
     688    1544   16197 file-system.ipynb
       3       6      81 file-system.ipynb.license
     464    1117   10935 grep-find.ipynb
       3       6      81 grep-find.ipynb.license
      22      57     540 index.rst
     378     873    8395 pipes-filters.ipynb
       3       6      81 pipes-filters.ipynb.license
     186     343    4479 shell-variables.ipynb
       3       6      81 shell-variables.ipynb.license
    2135    4849   50193 total

Now we write the number of characters in the file length.txt and then output the text with cat:

!wc -m *.* > length.txt

!cat length.txt

    9213 create-delete.ipynb
      81 create-delete.ipynb.license
   16125 file-system.ipynb
      81 file-system.ipynb.license
   10935 grep-find.ipynb
      81 grep-find.ipynb.license
     540 index.rst
    8395 pipes-filters.ipynb
      81 pipes-filters.ipynb.license
    4471 shell-variables.ipynb
      81 shell-variables.ipynb.license
   50084 total

We can also have the files sorted by the number of characters:

!sort -n length.txt

      81 create-delete.ipynb.license
      81 file-system.ipynb.license
      81 grep-find.ipynb.license
      81 pipes-filters.ipynb.license
      81 shell-variables.ipynb.license
     540 index.rst
    4471 shell-variables.ipynb
    8395 pipes-filters.ipynb
    9213 create-delete.ipynb
   10935 grep-find.ipynb
   16125 file-system.ipynb
   50084 total

!sort -n length.txt > sorted-length.txt

We can also overwrite the existing file:

!sort -n length.txt > length.txt

If we only want to know the total number of characters, in other words, only the output the last line, we can do this with tail:

!tail -n 1 length.txt

!echo amount of characters >> length.txt

> is used to overwrite a file while >> is used to append to a file.

!cat length.txt

amount of characters

Pipe |

You can connect commands with a pipe (|). In the following one-liner, we want to display the number of characters for the shortest file:

!wc -l *.* | sort -n | head

       1 length.txt
       3 create-delete.ipynb.license
       3 file-system.ipynb.license
       3 grep-find.ipynb.license
       3 pipes-filters.ipynb.license
       3 shell-variables.ipynb.license
      12 sorted-length.txt
      22 index.rst
     186 shell-variables.ipynb
     378 pipes-filters.ipynb

If we want to display the first lines of the main text (without the first three lines for the title):

!cat index.rst | head -n 5 | tail -n 2

Unix shell