{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Import notebooks\n", "\n", "To be able to develop more modularly, the import of notebooks is necessary. However, since notebooks are not Python files, they are not easy to import. Fortunately, Python provides some hooks for the import so that IPython notebooks can eventually be imported." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys\n", "import types" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import nbformat\n", "\n", "from IPython import get_ipython\n", "from IPython.core.interactiveshell import InteractiveShell" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Import hooks usually have two objects:\n", "\n", "* **Module Loader** that takes a module name (e.g. `IPython.display`) and returns a module\n", "* **Module Finder**, which finds out if a module is present and tells Python which *loader* to use\n", "\n", "But first, let’s write a method that a notebook will find using the fully qualified name and the optional path. E.g. `mypackage.foo` becomes `mypackage/foo.ipynb` and replaces `Foo_Bar` with `Foo Bar` if `Foo_Bar` doesn’t exist." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def find_notebook(fullname, path=None):\n", " name = fullname.rsplit(\".\", 1)[-1]\n", " if not path:\n", " path = [\"\"]\n", " for d in path:\n", " nb_path = os.path.join(d, name + \".ipynb\")\n", " if os.path.isfile(nb_path):\n", " return nb_path\n", " # let import Foo_Bar find \"Foo Bar.ipynb\"\n", " nb_path = nb_path.replace(\"_\", \" \")\n", " if os.path.isfile(nb_path):\n", " return nb_path" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Notebook Loader\n", "\n", "The Notebook Loader does the following three steps:\n", "\n", "1. Load the notebook document into memory\n", "2. Create an empty module\n", "3. Execute every cell in the module namespace\n", "\n", " Because IPython cells can have an extended syntax, `transform_cell` converts each cell to pure Python code before executing it." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "class NotebookLoader(object):\n", " \"\"\"Module Loader for IPython Notebooks\"\"\"\n", "\n", " def __init__(self, path=None):\n", " self.shell = InteractiveShell.instance()\n", " self.path = path\n", "\n", " def load_module(self, fullname):\n", " \"\"\"import a notebook as a module\"\"\"\n", " path = find_notebook(fullname, self.path)\n", "\n", " print(\"importing notebook from %s\" % path)\n", "\n", " # load the notebook object\n", " nb = nbformat.read(path, as_version=4)\n", "\n", " # create the module and add it to sys.modules\n", " # if name in sys.modules:\n", " # return sys.modules[name]\n", " mod = types.ModuleType(fullname)\n", " mod.__file__ = path\n", " mod.__loader__ = self\n", " mod.__dict__[\"get_ipython\"] = get_ipython\n", " sys.modules[fullname] = mod\n", "\n", " # extra work to ensure that magics that would affect the user_ns\n", " # magics that would affect the user_ns actually affect the\n", " # notebook module’s ns\n", " save_user_ns = self.shell.user_ns\n", " self.shell.user_ns = mod.__dict__\n", "\n", " try:\n", " for cell in nb.cells:\n", " if cell.cell_type == \"code\":\n", " # transform the input to executable Python\n", " code = self.shell.input_transformer_manager.transform_cell(\n", " cell.source\n", " )\n", " # run the code in the module\n", " exec(code, mod.__dict__)\n", " finally:\n", " self.shell.user_ns = save_user_ns\n", " return mod" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Notebook Finder\n", "\n", "The Finder is a simple object that indicates whether a notebook can be imported based on its file name and that returns the appropriate loader." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "class NotebookFinder(object):\n", " \"\"\"Module Finder finds the transformed IPython Notebook\"\"\"\n", "\n", " def __init__(self):\n", " self.loaders = {}\n", "\n", " def find_module(self, fullname, path=None):\n", " nb_path = find_notebook(fullname, path)\n", " if not nb_path:\n", " return\n", "\n", " key = path\n", " if path:\n", " # lists aren’t hashable\n", " key = os.path.sep.join(path)\n", "\n", " if key not in self.loaders:\n", " self.loaders[key] = NotebookLoader(path)\n", " return self.loaders[key] " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Register hook\n", "\n", "Now we register `NotebookFinder` with `sys.meta_path`:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "sys.meta_path.append(NotebookFinder())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Check" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now our notebook [mypackage/foo.ipynb](mypackage/foo.ipynb) should be importable with:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "importing notebook from /Users/veit/cusy/trn/Python4DataScience/docs/workspace/ipython/mypackage/foo.ipynb\n" ] } ], "source": [ "from mypackage import foo" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Is the Python method `bar` being executed?" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'bar'" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "foo.bar()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "… and the IPython syntax?" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['debugging.ipynb',\n", " 'display.ipynb',\n", " 'examples.ipynb',\n", " 'extensions.rst',\n", " 'importing.ipynb',\n", " 'index.rst',\n", " 'magics.ipynb',\n", " '\\x1b[34mmypackage\\x1b[m\\x1b[m',\n", " 'myscript.py',\n", " 'shell.ipynb',\n", " 'start.rst',\n", " '\\x1b[31mtab-completion-for-anything.png\\x1b[m\\x1b[m',\n", " '\\x1b[31mtab-completion-for-modules.png\\x1b[m\\x1b[m',\n", " '\\x1b[31mtab-completion-for-objects.png\\x1b[m\\x1b[m',\n", " '\\x1b[34munix-shell\\x1b[m\\x1b[m']" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "foo.dirlist()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reusable import hook\n", "\n", "The import hook can also easily be executed in other notebooks with" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "alert(\"This is an example of a Javascript warning displayed by IPython.\")" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "

markdown cell

\n", "
# `foo.ipynb`
\n", "

code cell

\n", "
def bar():\n",
       "    return "bar"\n",
       "
\n", "\n", "

code cell

\n", "
def dirlist():\n",
       "    listing = !ls\n",
       "    return listing\n",
       "
\n", "\n", "

code cell

\n", "
def whatsmyname():\n",
       "    return __name__\n",
       "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%run display.ipynb" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3.13 Kernel", "language": "python", "name": "python313" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.0" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": false, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 4 }