Essential Operations

This notebook covers sorting, validation, grouping, dissolving, and resegmentation of linearly referenced data.

import linref as lr

# Load sample datasets with LRS pre-configured
roadways = lr.datasets.load('roadways', set_lrs=True)
crashes = lr.datasets.load('crashes', set_lrs=True)

Standard Sorting

To ensure best performance, be sure to keep your data sorted. This can be achieved with the sort_standard method, which sorts data by your key columns (e.g., key_col) followed by your event measure columns (e.g., loc_col, beg_col, and end_col).

# Sort events upon loading to ensure best performance and correctness in downstream operations
resorted = roadways.sample(frac=1).lr.sort_standard()

print(resorted.head(3)[['route', 'beg', 'end']])

  route  beg   end
7   I-5  0.0   4.1
8   I-5  4.1   8.3
9   I-5  8.3  12.0

Remove Invalid Events

To avoid complications with some linref functionality, it is best to remove invalid events that may cause errors. This can be done with the drop_invalid_events method or by using the valid_events or invalid_events properties. These check for records which have missing data in the key columns or in event measure columns.

# Remove events with missing data to avoid downstream errors
has_invalid = roadways.copy()
has_invalid.loc[1, 'beg'] = None
all_valid = has_invalid.lr.drop_invalid_events()
print(all_valid.head(3)[['route', 'beg', 'end']])

    route  beg   end
0  US-101  0.0   2.5
2  US-101  5.0   7.8
3  US-101  7.8  10.5

Selecting Events by Group

If you simply need to access or analyze the dataframe, grouped by the unique groups represented in the LRS’s key column(s), you can do that with the get_group or iter_groups methods.

# Get events for a specific group
print(crashes.lr.get_group('SR-1')[['crash_id', 'route', 'loc']])

# Iterate over all groups to subset the data by unique groups
for group_name, group_df in crashes.lr.iter_groups():
    print(f"Group {group_name} has {len(group_df)} events")

# You can similarly get counts of events per group with the group_counts method
print(crashes.lr.group_counts())

    crash_id route   loc
6          7  SR-1  2.59
7          8  SR-1  7.62
8          9  SR-1  3.54
9         10  SR-1  6.00
11        12  SR-1  6.76
16        17  SR-1  3.74
17        18  SR-1  2.25
Group ('I-5',) has 4 events
Group ('SR-1',) has 7 events
Group ('US-101',) has 9 events
(I-5,)       4
(SR-1,)      7
(US-101,)    9
dtype: int64

Dissolving Events

Merge consecutive linear events within the same group based on shared event begin and end points. To dissolve by additional attributes, use the retain parameter which accepts a list of one or more dataframe column labels.

# Dissolve events, keeping specific columns
dissolved = roadways.lr.dissolve()

# Note that the dissolved_index column links back to the original events
view_columns = ['route', 'beg', 'end', 'dissolved_index']
print(dissolved[view_columns])

# Use the retain parameter to keep additional columns
print(roadways.lr.dissolve(retain=['speed_limit'])[view_columns + ['speed_limit']])

    route  beg   end dissolved_index
0     I-5  0.0  12.0       [7, 8, 9]
1    SR-1  0.0   9.8       [4, 5, 6]
2  US-101  0.0  10.5    [0, 1, 2, 3]
    route  beg   end dissolved_index  speed_limit
0     I-5  0.0  12.0       [7, 8, 9]           65
1    SR-1  0.0   3.2             [4]           35
2    SR-1  3.2   6.5             [5]           45
3    SR-1  6.5   9.8             [6]           55
4  US-101  0.0   2.5             [0]           45
5  US-101  7.8  10.5             [3]           45
6  US-101  2.5   7.8          [1, 2]           55

"The dissolve function will also merge geometries by default, retaining event measure information by upgrading geometries to be m-enabled."

# View the geometries created by the dissolve operation
print(dissolved.iloc[0].beg, dissolved.iloc[0].end)
print(dissolved.iloc[0].geometry)
print(dissolved.iloc[0].geometry_m)

0.0 12.0
LINESTRING (0 10, 4.1 11, 8.3 12.5, 12 14.2)
LINESTRING M (0.0 10.0 0.0, 4.1 11.0 4.1, 8.3 12.5 8.3, 12.0 14.2 12.0)

NOTE: The LINESTRING M shown here is using linref’s LineStringM class located in the geometry module. This is an implementation of m-enabled linear geometries that provides an extension of the existing shapely.LineString with the features needed for linref’s core functionality. Future versions of shapely are expected to provide native support for m-enabled linear geometries at which point linref will transition to their implementation. For now, please be cautious and review appropriate documentation if you are intending to use exports of these geometries in other programs. Use the WKT features of the LineStringM class and the linref LRS_Accessor class to support interoperability for now.

Resegmentation

Continuing with data engineering, we can take the dissolved roadways layer and resegment it to a fixed length using the resegment method:

# Create 5-mile segments
segmented = dissolved.lr.resegment(length=5)
print(segmented.lr.get_group('I-5')[['route', 'beg', 'end']])

  route   beg   end
0   I-5   0.0   5.0
1   I-5   5.0  10.0
2   I-5  10.0  12.0

Note that by default, the final segment is allowed to be shorter than others if the length of the parent route isn’t a multiple of the desired length. This behavior can be modified using the fill parameter:

# Various fill parameter options
resegmented = dissolved.lr.resegment(length=5, fill='cut') # cut (default)
print(resegmented.lr.get_group('I-5')[['route', 'beg', 'end']])

resegmented = dissolved.lr.resegment(length=5, fill='extend') # extend
print(resegmented.lr.get_group('I-5')[['route', 'beg', 'end']])

resegmented = dissolved.lr.resegment(length=5, fill='balance') # balance (cut when long, extend when short)
print(resegmented.lr.get_group('I-5')[['route', 'beg', 'end']])
# Not shown: right, left

  route   beg   end
0   I-5   0.0   5.0
1   I-5   5.0  10.0
2   I-5  10.0  12.0
  route  beg   end
0   I-5  0.0   5.0
1   I-5  5.0  12.0
  route  beg   end
0   I-5  0.0   5.0
1   I-5  5.0  12.0