{
"cells": [
{
"cell_type": "markdown",
"id": "9a8f2f85",
"metadata": {},
"source": [
"# `asyncio` example\n",
"\n",
"From IPython≥7.0 you can use `asyncio` directly in Jupyter Notebooks, see also [IPython 7.0, Async REPL](https://blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7).\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "a728279f",
"metadata": {},
"source": [
"If you get `RuntimeError: This event loop is already running`, [nest-asyncio] might help you.\n",
"\n",
"You can install the package with\n",
"\n",
"``` bash\n",
"$ uv add nest-asyncio\n",
"```\n",
"\n",
"You can then import it into your notebook and use it with:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "45aacd96",
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"\n",
"\n",
"nest_asyncio.apply()"
]
},
{
"cell_type": "markdown",
"id": "109900af",
"metadata": {},
"source": [
"\n",
"\n",
"**See also:**\n",
"\n",
"* [asyncio: We Did It Wrong](https://www.roguelynn.com/words/asyncio-we-did-it-wrong/) by Lynn Root\n",
"* [An Intro to asyncio](https://www.blog.pythonlibrary.org/2016/07/26/python-3-an-intro-to-asyncio/) by Mike Driscoll\n",
"* [Asyncio Coroutine Patterns: Beyond await](https://medium.com/python-pandemonium/asyncio-coroutine-patterns-beyond-await-a6121486656f) by Yeray Diaz\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "bc3e0b72",
"metadata": {},
"source": [
"## Simple *Hello world* example"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "479b2752",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello\n",
"world\n"
]
}
],
"source": [
"import asyncio\n",
"\n",
"\n",
"async def hello():\n",
" print(\"Hello\")\n",
" await asyncio.sleep(1)\n",
" print(\"world\")\n",
"\n",
"\n",
"await hello()"
]
},
{
"cell_type": "markdown",
"id": "c3d8764e",
"metadata": {},
"source": [
"## A little bit closer to a real world example"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "fc1761e7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"producing 1/10\n",
"producing 2/10\n",
"consuming 1\n",
"producing 3/10\n",
"consuming 2\n",
"producing 4/10\n",
"consuming 3\n",
"producing 5/10\n",
"consuming 4\n",
"producing 6/10\n",
"consuming 5\n",
"producing 7/10\n",
"consuming 6\n",
"producing 8/10\n",
"consuming 7\n",
"producing 9/10\n",
"consuming 8\n",
"producing 10/10\n",
"consuming 9\n",
"consuming 10\n"
]
}
],
"source": [
"import asyncio\n",
"import random\n",
"\n",
"\n",
"async def produce(queue, n):\n",
" for x in range(1, n + 1):\n",
" # produce an item\n",
" print(f\"producing {x}/{n}\")\n",
" # simulate i/o operation using sleep\n",
" await asyncio.sleep(random.random())\n",
" item = str(x)\n",
" # put the item in the queue\n",
" await queue.put(item)\n",
"\n",
" # indicate the producer is done\n",
" await queue.put(None)\n",
"\n",
"\n",
"async def consume(queue):\n",
" while True:\n",
" # wait for an item from the producer\n",
" item = await queue.get()\n",
" if item is None:\n",
" # the producer emits None to indicate that it is done\n",
" break\n",
"\n",
" # process the item\n",
" print(f\"consuming {item}\")\n",
" # simulate i/o operation using sleep\n",
" await asyncio.sleep(random.random())\n",
"\n",
"\n",
"loop = asyncio.get_event_loop()\n",
"queue = asyncio.Queue()\n",
"asyncio.ensure_future(produce(queue, 10), loop=loop)\n",
"loop.run_until_complete(consume(queue))"
]
},
{
"cell_type": "markdown",
"id": "80a9947d",
"metadata": {},
"source": [
"## Exception Handling\n",
"\n",
"\n",
"\n",
"**See also:**\n",
"\n",
"* [set_exception_handler](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.set_exception_handler)\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "54d7107d",
"metadata": {},
"outputs": [],
"source": [
"def main():\n",
" loop = asyncio.get_event_loop()\n",
" # May want to catch other signals too\n",
" signals = (signal.SIGHUP, signal.SIGTERM, signal.SIGINT)\n",
" for s in signals:\n",
" loop.add_signal_handler(\n",
" s, lambda s=s: asyncio.create_task(shutdown(loop, signal=s))\n",
" )\n",
" loop.set_exception_handler(handle_exception)\n",
" queue = asyncio.Queue()"
]
},
{
"cell_type": "markdown",
"id": "8b4270b5",
"metadata": {},
"source": [
"## Testing with `pytest`"
]
},
{
"cell_type": "markdown",
"id": "47a0aa94",
"metadata": {},
"source": [
"### Example:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "a196e119",
"metadata": {},
"outputs": [],
"source": [
"import pytest\n",
"\n",
"\n",
"@pytest.mark.asyncio\n",
"async def test_consume(mock_get, mock_queue, message, create_mock_coro):\n",
" mock_get.side_effect = [message, Exception(\"break while loop\")]\n",
"\n",
" with pytest.raises(Exception, match=\"break while loop\"):\n",
" await consume(mock_queue)"
]
},
{
"cell_type": "markdown",
"id": "1ea35d70",
"metadata": {},
"source": [
"### Third-party libraries\n",
"\n",
"* [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio) has helpful things like fixtures for `event_loop`, `unused_tcp_port`, and `unused_tcp_port_factory`; and the ability to create your own [asynchronous fixtures](https://pytest-asyncio.readthedocs.io/en/latest/reference/fixtures/index.html).\n",
"* [asynctest](https://asynctest.readthedocs.io/en/latest/index.html) has helpful tooling, including coroutine mocks and [exhaust_callbacks](https://asynctest.readthedocs.io/en/latest/asynctest.helpers.html#asynctest.helpers.exhaust_callbacks) so we don’t have to manually await tasks.\n",
"* [aiohttp](https://docs.aiohttp.org/en/stable/) has some really nice built-in test utilities."
]
},
{
"cell_type": "markdown",
"id": "8f9acdb7",
"metadata": {},
"source": [
"## Debugging\n",
"\n",
"`asyncio` already has a [debug mode](https://docs.python.org/3.6/library/asyncio-dev.html#debug-mode-of-asyncio) in the standard library. You can simply activate it with the `PYTHONASYNCIODEBUG` environment variable or in the code with `loop.set_debug(True)`."
]
},
{
"cell_type": "markdown",
"id": "6ef4d1c5",
"metadata": {},
"source": [
"### Using the debug mode to identify slow async calls\n",
"\n",
"`asyncio`’s debug mode has a tiny built-in profiler. When debug mode is on, `asyncio` will log any asynchronous calls that take longer than 100 milliseconds."
]
},
{
"cell_type": "markdown",
"id": "f4daa532",
"metadata": {},
"source": [
"### Debugging in oroduction with `aiodebug`\n",
"\n",
"[aiodebug](https://github.com/qntln/aiodebug) is a tiny library for monitoring and testing asyncio programs."
]
},
{
"cell_type": "markdown",
"id": "584b01ce",
"metadata": {},
"source": [
"#### Example"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "79686fa8",
"metadata": {},
"outputs": [],
"source": [
"from aiodebug import log_slow_callbacks\n",
"\n",
"\n",
"def main():\n",
" loop = asyncio.get_event_loop()\n",
" log_slow_callbacks.enable(0.05)"
]
},
{
"cell_type": "markdown",
"id": "dac368ac",
"metadata": {},
"source": [
"## Logging\n",
"\n",
"[aiologger](https://github.com/async-worker/aiologger) allows non-blocking logging."
]
},
{
"cell_type": "markdown",
"id": "050af8ac",
"metadata": {},
"source": [
"## Asynchronous Widgets\n",
"\n",
"\n",
"\n",
"**See also:**\n",
"\n",
"* [Asynchronous Widgets](https://ipywidgets.readthedocs.io/en/stable/examples/Widget%20Asynchronous.html)\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "6925cd9d",
"metadata": {},
"outputs": [],
"source": [
"def wait_for_change(widget, value):\n",
" future = asyncio.Future()\n",
"\n",
" def getvalue(change):\n",
" # make the new value available\n",
" future.set_result(change.new)\n",
" widget.unobserve(getvalue, value)\n",
"\n",
" widget.observe(getvalue, value)\n",
" return future"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "fb57d28a",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d7270d23d2fd4a7097d28183a469b0f0",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"IntSlider(value=0)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"did work 0\n",
"async function continued with value 1\n",
"did work 1\n",
"async function continued with value 2\n",
"did work 2\n",
"async function continued with value 3\n",
"did work 3\n",
"async function continued with value 4\n",
"did work 4\n",
"async function continued with value 5\n",
"did work 5\n",
"async function continued with value 6\n",
"did work 6\n",
"async function continued with value 7\n",
"did work 7\n",
"async function continued with value 8\n",
"did work 8\n",
"async function continued with value 9\n",
"did work 9\n",
"async function continued with value 10\n"
]
}
],
"source": [
"from ipywidgets import IntSlider\n",
"\n",
"\n",
"slider = IntSlider()\n",
"\n",
"\n",
"async def f():\n",
" for i in range(10):\n",
" print(f\"did work {i}\")\n",
" x = await wait_for_change(slider, \"value\")\n",
" print(f\"async function continued with value {x}\")\n",
"\n",
"\n",
"asyncio.ensure_future(f())\n",
"\n",
"slider"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.13 Kernel",
"language": "python",
"name": "python313"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.0"
},
"latex_envs": {
"LaTeX_envs_menu_present": true,
"autoclose": false,
"autocomplete": true,
"bibliofile": "biblio.bib",
"cite_by": "apalike",
"current_citInitial": 1,
"eqLabelWithNumbers": true,
"eqNumInitial": 1,
"hotkeys": {
"equation": "Ctrl-E",
"itemize": "Ctrl-I"
},
"labels_anchors": false,
"latex_user_defs": false,
"report_style_numbering": false,
"user_envs_cfg": false
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {
"5bf031e8df594b84ac0e66b071d65823": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "2.0.0",
"model_name": "LayoutModel",
"state": {}
},
"8e07d4f66b794a62b1a5c316c2c9dcf7": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "2.0.0",
"model_name": "SliderStyleModel",
"state": {
"description_width": ""
}
},
"d7270d23d2fd4a7097d28183a469b0f0": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "2.0.0",
"model_name": "IntSliderModel",
"state": {
"behavior": "drag-tap",
"layout": "IPY_MODEL_5bf031e8df594b84ac0e66b071d65823",
"style": "IPY_MODEL_8e07d4f66b794a62b1a5c316c2c9dcf7",
"value": 10
}
}
},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}