{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5cf77b40",
   "metadata": {},
   "source": [
    "# Essential Operations\n",
    "\n",
    "This notebook covers sorting, validation, grouping, dissolving, and resegmentation of linearly referenced data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "eab01bf1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:35.434035Z",
     "iopub.status.busy": "2026-05-29T16:42:35.433742Z",
     "iopub.status.idle": "2026-05-29T16:42:36.006405Z",
     "shell.execute_reply": "2026-05-29T16:42:36.005608Z"
    }
   },
   "outputs": [],
   "source": [
    "import linref as lr\n",
    "\n",
    "# Load sample datasets with LRS pre-configured\n",
    "roadways = lr.datasets.load('roadways', set_lrs=True)\n",
    "crashes = lr.datasets.load('crashes', set_lrs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83dea927",
   "metadata": {},
   "source": [
    "## Standard Sorting\n",
    "\n",
    "To ensure best performance, be sure to keep your data sorted. This can be achieved with the `sort_standard` method, which sorts data by your key columns (e.g., `key_col`) followed by your event measure columns (e.g., `loc_col`, `beg_col`, and `end_col`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3aa2ab69",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:36.008675Z",
     "iopub.status.busy": "2026-05-29T16:42:36.008461Z",
     "iopub.status.idle": "2026-05-29T16:42:36.016711Z",
     "shell.execute_reply": "2026-05-29T16:42:36.016223Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  route  beg   end\n",
      "7   I-5  0.0   4.1\n",
      "8   I-5  4.1   8.3\n",
      "9   I-5  8.3  12.0\n"
     ]
    }
   ],
   "source": [
    "# Sort events upon loading to ensure best performance and correctness in downstream operations\n",
    "resorted = roadways.sample(frac=1).lr.sort_standard()\n",
    "\n",
    "print(resorted.head(3)[['route', 'beg', 'end']])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3090f0f7",
   "metadata": {},
   "source": [
    "## Remove Invalid Events\n",
    "\n",
    "To avoid complications with some linref functionality, it is best to remove invalid events that may cause errors. This can be done with the `drop_invalid_events` method or by using the `valid_events` or `invalid_events` properties. These check for records which have missing data in the key columns or in event measure columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bb0cdba7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:36.018113Z",
     "iopub.status.busy": "2026-05-29T16:42:36.017919Z",
     "iopub.status.idle": "2026-05-29T16:42:36.023422Z",
     "shell.execute_reply": "2026-05-29T16:42:36.022912Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    route  beg   end\n",
      "0  US-101  0.0   2.5\n",
      "2  US-101  5.0   7.8\n",
      "3  US-101  7.8  10.5\n"
     ]
    }
   ],
   "source": [
    "# Remove events with missing data to avoid downstream errors\n",
    "has_invalid = roadways.copy()\n",
    "has_invalid.loc[1, 'beg'] = None\n",
    "all_valid = has_invalid.lr.drop_invalid_events()\n",
    "print(all_valid.head(3)[['route', 'beg', 'end']])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "992af052",
   "metadata": {},
   "source": [
    "## Selecting Events by Group\n",
    "\n",
    "If you simply need to access or analyze the dataframe, grouped by the unique groups represented in the LRS's key column(s), you can do that with the `get_group` or `iter_groups` methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "271a9bb3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:36.024721Z",
     "iopub.status.busy": "2026-05-29T16:42:36.024516Z",
     "iopub.status.idle": "2026-05-29T16:42:36.035594Z",
     "shell.execute_reply": "2026-05-29T16:42:36.035038Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    crash_id route   loc\n",
      "6          7  SR-1  2.59\n",
      "7          8  SR-1  7.62\n",
      "8          9  SR-1  3.54\n",
      "9         10  SR-1  6.00\n",
      "11        12  SR-1  6.76\n",
      "16        17  SR-1  3.74\n",
      "17        18  SR-1  2.25\n",
      "Group ('I-5',) has 4 events\n",
      "Group ('SR-1',) has 7 events\n",
      "Group ('US-101',) has 9 events\n",
      "(I-5,)       4\n",
      "(SR-1,)      7\n",
      "(US-101,)    9\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Get events for a specific group\n",
    "print(crashes.lr.get_group('SR-1')[['crash_id', 'route', 'loc']])\n",
    "\n",
    "# Iterate over all groups to subset the data by unique groups\n",
    "for group_name, group_df in crashes.lr.iter_groups():\n",
    "    print(f\"Group {group_name} has {len(group_df)} events\")\n",
    "\n",
    "# You can similarly get counts of events per group with the group_counts method\n",
    "print(crashes.lr.group_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17ef51e0",
   "metadata": {},
   "source": [
    "## Dissolving Events\n",
    "\n",
    "Merge consecutive linear events within the same group based on shared event begin and end points. To dissolve by additional attributes, use the `retain` parameter which accepts a list of one or more dataframe column labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "bce780b1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:36.037250Z",
     "iopub.status.busy": "2026-05-29T16:42:36.036954Z",
     "iopub.status.idle": "2026-05-29T16:42:36.052410Z",
     "shell.execute_reply": "2026-05-29T16:42:36.051952Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    route  beg   end dissolved_index\n",
      "0     I-5  0.0  12.0       [7, 8, 9]\n",
      "1    SR-1  0.0   9.8       [4, 5, 6]\n",
      "2  US-101  0.0  10.5    [0, 1, 2, 3]\n",
      "    route  beg   end dissolved_index  speed_limit\n",
      "0     I-5  0.0  12.0       [7, 8, 9]           65\n",
      "1    SR-1  0.0   3.2             [4]           35\n",
      "2    SR-1  3.2   6.5             [5]           45\n",
      "3    SR-1  6.5   9.8             [6]           55\n",
      "4  US-101  0.0   2.5             [0]           45\n",
      "5  US-101  7.8  10.5             [3]           45\n",
      "6  US-101  2.5   7.8          [1, 2]           55\n"
     ]
    }
   ],
   "source": [
    "# Dissolve events, keeping specific columns\n",
    "dissolved = roadways.lr.dissolve()\n",
    "\n",
    "# Note that the dissolved_index column links back to the original events\n",
    "view_columns = ['route', 'beg', 'end', 'dissolved_index']\n",
    "print(dissolved[view_columns])\n",
    "\n",
    "# Use the retain parameter to keep additional columns\n",
    "print(roadways.lr.dissolve(retain=['speed_limit'])[view_columns + ['speed_limit']])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d5557d6",
   "metadata": {},
   "source": [
    "    \"The dissolve function will also merge geometries by default, retaining event measure information by upgrading geometries to be m-enabled.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7bcbd0e4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:36.054088Z",
     "iopub.status.busy": "2026-05-29T16:42:36.053829Z",
     "iopub.status.idle": "2026-05-29T16:42:36.057048Z",
     "shell.execute_reply": "2026-05-29T16:42:36.056669Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0 12.0\n",
      "LINESTRING (0 10, 4.1 11, 8.3 12.5, 12 14.2)\n",
      "LINESTRING M (0.0 10.0 0.0, 4.1 11.0 4.1, 8.3 12.5 8.3, 12.0 14.2 12.0)\n"
     ]
    }
   ],
   "source": [
    "# View the geometries created by the dissolve operation\n",
    "print(dissolved.iloc[0].beg, dissolved.iloc[0].end)\n",
    "print(dissolved.iloc[0].geometry)\n",
    "print(dissolved.iloc[0].geometry_m)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfb38d7f",
   "metadata": {},
   "source": [
    "NOTE: The LINESTRING M shown here is using linref's `LineStringM` class located in the geometry module. This is an implementation of m-enabled linear geometries that provides an extension of the existing `shapely.LineString` with the features needed for linref's core functionality. Future versions of shapely are expected to provide native support for m-enabled linear geometries at which point linref will transition to their implementation. For now, please be cautious and review appropriate documentation if you are intending to use exports of these geometries in other programs. Use the WKT features of the `LineStringM` class and the linref `LRS_Accessor` class to support interoperability for now."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dbce76a",
   "metadata": {},
   "source": [
    "## Resegmentation\n",
    "\n",
    "Continuing with data engineering, we can take the dissolved roadways layer and resegment it to a fixed length using the `resegment` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3c84fe67",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:36.058474Z",
     "iopub.status.busy": "2026-05-29T16:42:36.058332Z",
     "iopub.status.idle": "2026-05-29T16:42:36.072236Z",
     "shell.execute_reply": "2026-05-29T16:42:36.071784Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  route   beg   end\n",
      "0   I-5   0.0   5.0\n",
      "1   I-5   5.0  10.0\n",
      "2   I-5  10.0  12.0\n"
     ]
    }
   ],
   "source": [
    "# Create 5-mile segments\n",
    "segmented = dissolved.lr.resegment(length=5)\n",
    "print(segmented.lr.get_group('I-5')[['route', 'beg', 'end']])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "618122e9",
   "metadata": {},
   "source": [
    "Note that by default, the final segment is allowed to be shorter than others if the length of the parent route isn't a multiple of the desired length. This behavior can be modified using the `fill` parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "69d4ad12",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-29T16:42:36.073988Z",
     "iopub.status.busy": "2026-05-29T16:42:36.073839Z",
     "iopub.status.idle": "2026-05-29T16:42:36.102645Z",
     "shell.execute_reply": "2026-05-29T16:42:36.101950Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  route   beg   end\n",
      "0   I-5   0.0   5.0\n",
      "1   I-5   5.0  10.0\n",
      "2   I-5  10.0  12.0\n",
      "  route  beg   end\n",
      "0   I-5  0.0   5.0\n",
      "1   I-5  5.0  12.0\n",
      "  route  beg   end\n",
      "0   I-5  0.0   5.0\n",
      "1   I-5  5.0  12.0\n"
     ]
    }
   ],
   "source": [
    "# Various fill parameter options\n",
    "resegmented = dissolved.lr.resegment(length=5, fill='cut') # cut (default)\n",
    "print(resegmented.lr.get_group('I-5')[['route', 'beg', 'end']])\n",
    "\n",
    "resegmented = dissolved.lr.resegment(length=5, fill='extend') # extend\n",
    "print(resegmented.lr.get_group('I-5')[['route', 'beg', 'end']])\n",
    "\n",
    "resegmented = dissolved.lr.resegment(length=5, fill='balance') # balance (cut when long, extend when short)\n",
    "print(resegmented.lr.get_group('I-5')[['route', 'beg', 'end']])\n",
    "# Not shown: right, left"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv (3.12.3)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}