{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3e1ff7c5",
   "metadata": {},
   "source": [
    "# Pipes and filters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a6c2996",
   "metadata": {},
   "source": [
    "`ls` shows all files and directories at this point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8841439a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "create-delete.ipynb           index.rst\n",
      "create-delete.ipynb.license   pipes-filters.ipynb\n",
      "file-system.ipynb             pipes-filters.ipynb.license\n",
      "file-system.ipynb.license     shell-variables.ipynb\n",
      "grep-find.ipynb               shell-variables.ipynb.license\n",
      "grep-find.ipynb.license\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19ef1512",
   "metadata": {},
   "source": [
    "With `*.rst` we restrict the results to all files with the suffix `.rst`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a708e558",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "create-delete.ipynb           index.rst\n",
      "create-delete.ipynb.license   pipes-filters.ipynb\n",
      "file-system.ipynb             pipes-filters.ipynb.license\n",
      "file-system.ipynb.license     shell-variables.ipynb\n",
      "grep-find.ipynb               shell-variables.ipynb.license\n",
      "grep-find.ipynb.license\n"
     ]
    }
   ],
   "source": [
    "!ls *.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a543f782",
   "metadata": {},
   "source": [
    "We can also output only the number of lines, words and characters in these documents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "73cf6ec2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     382     885    9242 create-delete.ipynb\n",
      "       3       6      81 create-delete.ipynb.license\n",
      "     688    1544   16197 file-system.ipynb\n",
      "       3       6      81 file-system.ipynb.license\n",
      "     464    1117   10935 grep-find.ipynb\n",
      "       3       6      81 grep-find.ipynb.license\n",
      "      22      57     540 index.rst\n",
      "     378     873    8395 pipes-filters.ipynb\n",
      "       3       6      81 pipes-filters.ipynb.license\n",
      "     186     343    4479 shell-variables.ipynb\n",
      "       3       6      81 shell-variables.ipynb.license\n",
      "    2135    4849   50193 total\n"
     ]
    }
   ],
   "source": [
    "!wc *.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4880b3bc",
   "metadata": {},
   "source": [
    "Now we write the number of characters in the file `length.txt` and then output the text with `cat`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8d8c65ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "!wc -m *.* > length.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6575252c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    9213 create-delete.ipynb\n",
      "      81 create-delete.ipynb.license\n",
      "   16125 file-system.ipynb\n",
      "      81 file-system.ipynb.license\n",
      "   10935 grep-find.ipynb\n",
      "      81 grep-find.ipynb.license\n",
      "     540 index.rst\n",
      "    8395 pipes-filters.ipynb\n",
      "      81 pipes-filters.ipynb.license\n",
      "    4471 shell-variables.ipynb\n",
      "      81 shell-variables.ipynb.license\n",
      "   50084 total\n"
     ]
    }
   ],
   "source": [
    "!cat length.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6f20e5e",
   "metadata": {},
   "source": [
    "We can also have the files sorted by the number of characters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b23b6199",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      81 create-delete.ipynb.license\n",
      "      81 file-system.ipynb.license\n",
      "      81 grep-find.ipynb.license\n",
      "      81 pipes-filters.ipynb.license\n",
      "      81 shell-variables.ipynb.license\n",
      "     540 index.rst\n",
      "    4471 shell-variables.ipynb\n",
      "    8395 pipes-filters.ipynb\n",
      "    9213 create-delete.ipynb\n",
      "   10935 grep-find.ipynb\n",
      "   16125 file-system.ipynb\n",
      "   50084 total\n"
     ]
    }
   ],
   "source": [
    "!sort -n length.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6206c676",
   "metadata": {},
   "outputs": [],
   "source": [
    "!sort -n length.txt > sorted-length.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "663f103b",
   "metadata": {},
   "source": [
    "We can also overwrite the existing file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2f7e634d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!sort -n length.txt > length.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9c3fc7b",
   "metadata": {},
   "source": [
    "If we only want to know the total number of characters, in other words, only the output the last line, we can do this with `tail`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "9131f50d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!tail -n 1 length.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "66ff88ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "!echo amount of characters >> length.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7e3ba9e",
   "metadata": {},
   "source": [
    "`>`  is used to overwrite a file while `>>` is used to append to a file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "5796d337",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "amount of characters\n"
     ]
    }
   ],
   "source": [
    "!cat length.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f6e36a3",
   "metadata": {},
   "source": [
    "### Pipe `|`\n",
    "\n",
    "You can connect commands with a pipe (`|`). In the following one-liner, we want to display the number of characters for the shortest file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c0a7259a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       1 length.txt\n",
      "       3 create-delete.ipynb.license\n",
      "       3 file-system.ipynb.license\n",
      "       3 grep-find.ipynb.license\n",
      "       3 pipes-filters.ipynb.license\n",
      "       3 shell-variables.ipynb.license\n",
      "      12 sorted-length.txt\n",
      "      22 index.rst\n",
      "     186 shell-variables.ipynb\n",
      "     378 pipes-filters.ipynb\n"
     ]
    }
   ],
   "source": [
    "!wc -l *.* | sort -n | head"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1f9ca10",
   "metadata": {},
   "source": [
    "If we want to display the first lines of the main text (without the first three lines for the title):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d4380f5a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Unix shell\n"
     ]
    }
   ],
   "source": [
    "!cat index.rst | head -n 5 | tail -n 2"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.13 Kernel",
   "language": "python",
   "name": "python313"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}