Pipes and filters

ls shows all files and directories at this point.

[1]:
!ls
create-delete.ipynb   grep-find.ipynb       shell-variables.ipynb
dvc.list              index.rst
file-system.ipynb     pipes-filters.ipynb

With *.rst we restrict the results to all files with the suffix .rst:

[2]:
!ls *.rst
index.rst

We can also output only the number of lines, words and characters in these documents:

[3]:
!wc *.rst
      18      48     450 index.rst

Now we write the number of characters in the file length.txt and then output the text with cat:

[4]:
!wc -m *.rst > length.txt
[5]:
!cat length.txt
     450 index.rst

We can also have the files sorted by the number of characters:

[6]:
!sort -n length.txt
     450 index.rst
[7]:
!sort -n length.txt > sorted-length.txt

We can also overwrite the existing file:

[8]:
!sort -n length.txt > length.txt

If we only want to know the total number of characters, i.e. only output the last line, we can do this with tail:

[9]:
!tail -n 1 length.txt

> is used to overwrite a file while >> is used to append to a file.

[10]:
!echo amount of characters >> length.txt
[11]:
!cat length.txt
amount of characters

Pipe |

You can connect commands with a pipe (|). In the following one-liner, we want to display the number of characters for the shortest file:

[12]:
!wc -l *.rst | sort -n | head
      18 index.rst

If we want to display the first lines of the main text (without the first three lines for the title):

[13]:
!cat index.rst | head -n 5 | tail -n 2
Any command on the command line will also work in Jupyter Notebooks if prefixed
with ``!``. The results can then interact with the Jupyter namespace, see