{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "35cc9e67",
   "metadata": {},
   "source": [
    "# Subdividing and categorising data\n",
    "\n",
    "Continuous data is often divided into domains or otherwise grouped for analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "625f4a8b",
   "metadata": {},
   "source": [
    "Suppose you have data on a group of people in a study that you want to divide into discrete age groups. For this, we generate a dataframe with 250 entries between `0` and `99`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "fe17e156",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.250228Z",
     "iopub.status.busy": "2026-05-21T13:00:07.249985Z",
     "iopub.status.idle": "2026-05-21T13:00:07.574238Z",
     "shell.execute_reply": "2026-05-21T13:00:07.573964Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.250209Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>245</th>\n",
       "      <td>59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>246</th>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>247</th>\n",
       "      <td>88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>248</th>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>249</th>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>250 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Age\n",
       "0     40\n",
       "1     33\n",
       "2      8\n",
       "3     90\n",
       "4     98\n",
       "..   ...\n",
       "245   59\n",
       "246   46\n",
       "247   88\n",
       "248   10\n",
       "249   37\n",
       "\n",
       "[250 rows x 1 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "\n",
    "rng = np.random.default_rng()\n",
    "ages = rng.integers(0, 99, 250)\n",
    "df = pd.DataFrame({\"Age\": ages})\n",
    "\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8717607a",
   "metadata": {},
   "source": [
    "Afterwards, pandas offers us a simple way to divide the results into ten ranges with [pandas.cut](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.cut.html). To get only whole years, we additionally set `precision=0`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2c89e759",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.574846Z",
     "iopub.status.busy": "2026-05-21T13:00:07.574683Z",
     "iopub.status.idle": "2026-05-21T13:00:07.578315Z",
     "shell.execute_reply": "2026-05-21T13:00:07.578064Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.574837Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(39.0, 49.0], (29.0, 39.0], (-0.1, 10.0], (88.0, 98.0], (88.0, 98.0], ..., (59.0, 69.0], (39.0, 49.0], (78.0, 88.0], (10.0, 20.0], (29.0, 39.0]]\n",
       "Length: 250\n",
       "Categories (10, interval[float64, right]): [(-0.1, 10.0] < (10.0, 20.0] < (20.0, 29.0] < (29.0, 39.0] ... (59.0, 69.0] < (69.0, 78.0] < (78.0, 88.0] < (88.0, 98.0]]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cats = pd.cut(ages, 10, precision=0)\n",
    "\n",
    "cats"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40135b6f",
   "metadata": {},
   "source": [
    "With [pandas.Categorical.categories](https://pandas.pydata.org/docs/reference/api/pandas.Categorical.categories.html) you can display the categories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "48c787f3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.578841Z",
     "iopub.status.busy": "2026-05-21T13:00:07.578710Z",
     "iopub.status.idle": "2026-05-21T13:00:07.581043Z",
     "shell.execute_reply": "2026-05-21T13:00:07.580744Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.578833Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "IntervalIndex([(-0.1, 10.0], (10.0, 20.0], (20.0, 29.0], (29.0, 39.0],\n",
       "               (39.0, 49.0], (49.0, 59.0], (59.0, 69.0], (69.0, 78.0],\n",
       "               (78.0, 88.0], (88.0, 98.0]],\n",
       "              dtype='interval[float64, right]')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cats.categories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5987f98",
   "metadata": {},
   "source": [
    "… or even just a single category:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "59da3a5e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.581512Z",
     "iopub.status.busy": "2026-05-21T13:00:07.581443Z",
     "iopub.status.idle": "2026-05-21T13:00:07.583907Z",
     "shell.execute_reply": "2026-05-21T13:00:07.583578Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.581505Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Interval(-0.1, 10.0, closed='right')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cats.categories[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08fd3777",
   "metadata": {},
   "source": [
    "With [pandas.Categorical.codes](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Categorical.codes.html) you can display an array where for each value the corresponding category is shown:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0df59a92",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.584365Z",
     "iopub.status.busy": "2026-05-21T13:00:07.584286Z",
     "iopub.status.idle": "2026-05-21T13:00:07.586762Z",
     "shell.execute_reply": "2026-05-21T13:00:07.586470Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.584358Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([4, 3, 0, 9, 9, 1, 3, 4, 1, 8, 9, 6, 8, 7, 1, 3, 4, 2, 1, 2, 7, 0,\n",
       "       8, 9, 1, 4, 2, 9, 4, 5, 8, 6, 3, 8, 6, 5, 5, 9, 0, 0, 4, 8, 0, 5,\n",
       "       5, 4, 0, 5, 9, 6, 2, 6, 9, 6, 4, 4, 8, 6, 7, 7, 2, 2, 4, 6, 8, 0,\n",
       "       6, 8, 3, 4, 6, 0, 1, 7, 7, 6, 3, 9, 6, 2, 0, 7, 8, 0, 7, 2, 6, 0,\n",
       "       2, 3, 4, 1, 5, 0, 7, 0, 0, 9, 0, 7, 6, 0, 0, 1, 1, 6, 5, 2, 1, 2,\n",
       "       5, 0, 0, 0, 0, 0, 1, 2, 0, 6, 8, 4, 3, 6, 6, 5, 9, 6, 1, 1, 7, 5,\n",
       "       3, 7, 6, 1, 3, 3, 8, 6, 1, 4, 4, 9, 7, 2, 7, 5, 2, 7, 5, 5, 4, 5,\n",
       "       5, 9, 4, 9, 3, 7, 3, 3, 1, 2, 0, 4, 0, 4, 0, 3, 5, 8, 9, 0, 5, 7,\n",
       "       8, 2, 4, 0, 4, 3, 3, 6, 0, 9, 7, 7, 3, 1, 0, 1, 7, 8, 0, 2, 3, 5,\n",
       "       3, 3, 1, 2, 5, 8, 0, 3, 1, 5, 0, 3, 9, 2, 0, 1, 0, 9, 4, 6, 3, 8,\n",
       "       7, 1, 9, 1, 0, 3, 1, 8, 4, 0, 5, 2, 6, 9, 2, 5, 5, 4, 9, 1, 3, 6,\n",
       "       9, 0, 6, 6, 4, 8, 1, 3], dtype=int8)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cats.codes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "347712ac",
   "metadata": {},
   "source": [
    "With `value_counts` we can now look at how the number is distributed among the individual areas:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "048cc13b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.588326Z",
     "iopub.status.busy": "2026-05-21T13:00:07.588233Z",
     "iopub.status.idle": "2026-05-21T13:00:07.591687Z",
     "shell.execute_reply": "2026-05-21T13:00:07.591469Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.588318Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(-0.1, 10.0]    38\n",
       "(29.0, 39.0]    27\n",
       "(59.0, 69.0]    27\n",
       "(10.0, 20.0]    26\n",
       "(39.0, 49.0]    25\n",
       "(49.0, 59.0]    24\n",
       "(88.0, 98.0]    22\n",
       "(20.0, 29.0]    21\n",
       "(69.0, 78.0]    21\n",
       "(78.0, 88.0]    19\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(cats).value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4f58496",
   "metadata": {},
   "source": [
    "It is striking that the age ranges do not contain an equal number of years, but with `20.0, 29.0` and `69.0, 78.0` two ranges contain only 9 years. This is due to the fact that the age range only extends from `0` to `98`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "062036d8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.592141Z",
     "iopub.status.busy": "2026-05-21T13:00:07.592064Z",
     "iopub.status.idle": "2026-05-21T13:00:07.594543Z",
     "shell.execute_reply": "2026-05-21T13:00:07.594303Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.592134Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Age    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9579e708",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.595176Z",
     "iopub.status.busy": "2026-05-21T13:00:07.595014Z",
     "iopub.status.idle": "2026-05-21T13:00:07.597566Z",
     "shell.execute_reply": "2026-05-21T13:00:07.597246Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.595165Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Age    98\n",
       "dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6fc3deb",
   "metadata": {},
   "source": [
    "With [pandas.qcut](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.qcut.html), on the other hand, the set is divided into areas that are approximately the same size:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "fb4d41b1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.598061Z",
     "iopub.status.busy": "2026-05-21T13:00:07.597953Z",
     "iopub.status.idle": "2026-05-21T13:00:07.600985Z",
     "shell.execute_reply": "2026-05-21T13:00:07.600676Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.598054Z"
    }
   },
   "outputs": [],
   "source": [
    "cats = pd.qcut(ages, 10, precision=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8653757b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.601497Z",
     "iopub.status.busy": "2026-05-21T13:00:07.601419Z",
     "iopub.status.idle": "2026-05-21T13:00:07.604675Z",
     "shell.execute_reply": "2026-05-21T13:00:07.604351Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.601490Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(44.0, 55.0]    31\n",
       "(-1.0, 7.0]     28\n",
       "(76.0, 88.0]    27\n",
       "(25.0, 34.0]    26\n",
       "(34.0, 44.0]    25\n",
       "(7.0, 15.0]     24\n",
       "(65.0, 76.0]    24\n",
       "(15.0, 25.0]    23\n",
       "(88.0, 98.0]    22\n",
       "(55.0, 65.0]    20\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(cats).value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1f75798",
   "metadata": {},
   "source": [
    "If we want to ensure that each age group actually includes exactly ten years, we can specify this directly with [pandas.Categorical](https://pandas.pydata.org/docs/reference/api/pandas.Categorical.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "3be29378",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.605103Z",
     "iopub.status.busy": "2026-05-21T13:00:07.605016Z",
     "iopub.status.idle": "2026-05-21T13:00:07.607953Z",
     "shell.execute_reply": "2026-05-21T13:00:07.607636Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.605095Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['0 - 9', '10 - 19', '20 - 29', '30 - 39', '40 - 49', '50 - 59',\n",
       "       '60 - 69', '70 - 79', '80 - 89', '90 - 99'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "age_groups = [f\"{i} - {i + 9}\" for i in range(0, 99, 10)]\n",
    "cats = pd.Categorical(age_groups)\n",
    "\n",
    "cats.categories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b281dfd0",
   "metadata": {},
   "source": [
    "For grouping we can now use [pandas.cut](https://pandas.pydata.org/docs/reference/api/pandas.cut.html). However, the number of labels must be one less than the number of edges:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f5180bc8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-21T13:00:07.608417Z",
     "iopub.status.busy": "2026-05-21T13:00:07.608346Z",
     "iopub.status.idle": "2026-05-21T13:00:07.611990Z",
     "shell.execute_reply": "2026-05-21T13:00:07.611762Z",
     "shell.execute_reply.started": "2026-05-21T13:00:07.608411Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Age group</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>40</td>\n",
       "      <td>40 - 49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>33</td>\n",
       "      <td>30 - 39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>0 - 9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>90</td>\n",
       "      <td>90 - 99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>98</td>\n",
       "      <td>90 - 99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>245</th>\n",
       "      <td>59</td>\n",
       "      <td>50 - 59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>246</th>\n",
       "      <td>46</td>\n",
       "      <td>40 - 49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>247</th>\n",
       "      <td>88</td>\n",
       "      <td>80 - 89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>248</th>\n",
       "      <td>10</td>\n",
       "      <td>10 - 19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>249</th>\n",
       "      <td>37</td>\n",
       "      <td>30 - 39</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>250 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Age Age group\n",
       "0     40   40 - 49\n",
       "1     33   30 - 39\n",
       "2      8     0 - 9\n",
       "3     90   90 - 99\n",
       "4     98   90 - 99\n",
       "..   ...       ...\n",
       "245   59   50 - 59\n",
       "246   46   40 - 49\n",
       "247   88   80 - 89\n",
       "248   10   10 - 19\n",
       "249   37   30 - 39\n",
       "\n",
       "[250 rows x 2 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"Age group\"] = pd.cut(df.Age, range(0, 101, 10), right=False, labels=cats)\n",
    "\n",
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.13 Kernel",
   "language": "python",
   "name": "python313"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}