9  Vector Geoprocessing with GeoPandas

9.1 Topics Covered

  1. Read and write geospatial vector data with GeoPandas
  2. Apply general data wrangling and summarization to vector geospatial data using Pandas methods
  3. Work with coordinate reference systems and bounding boxes
  4. Convert between geometry types and generate bounding geometries
  5. Perform proximity analysis/buffering
  6. Perform geoprocessing and vector overlay
  7. Summarize point and line features relative to polygons
  8. Geometry simplification, spatial sampling, and tessellation

9.2 Introduction

9.2.1 Overview of GeoPandas

In this chapter, we explore a wide variety of techniques for working with and analyzing vector geospatial data in Python. Specifically, we focus on methods made available in the GeoPandas library. GeoPandas extends Pandas (Chapter 4) by adding the GeoDataFrame, which is a DataFrame with a special column, generally named “geometry”, that stores the geometric data. If you have worked with PostgreSQL and PostGIS, you will observe lots of similarities. Both make use of an object-based vector data model where the geometry and attributes are stored in the same table. In other words, the geometric data are simply stored as another column in the table alongside feature attributes. One benefit of this structure is that we can perform the same data preprocessing, manipulation, wrangling, and querying techniques on GeoDataFrames that we have already used for Pandas DataFrames.

Each value in the geometry column is a geometry object from the Shapely library. GeoPandas and Shapely offer several geometry types, as summarized in Table 9.1. Other than Point, LineString, and Polygon features, it is possible to create MultiPoint, MultiLineString, and MultiPolygon features where multiple, and potentially noncontiguous, geographic features are treated as one record in the table. Further, it is not required that all records or features in a GeoDataFrame have the same geometry type. The GeometryCollection type allows for a single record to hold multiple geometry types. The geom_type property returns the geometry type of each feature, and the explode() method converts multipart features to single-part features. Geometric errors can be detected using the is_valid property while the make_valid() method allows for correcting geometric errors. Note that corrections may not always be successful (only certain types of geometric errors can be detected and corrected).

Table 9.1. Shapely and GeoPandas geometry types.
Geometry Type Description
Point Each point treated as a single record
MultiPoint Allows multiple points to be treated like a single record
LineString Series of coordinates connected as a line
MultiLineString Allows for multiple lines to be treated as one record
Polygon Series of coordinates forming a closed, areal shape
MultiPolygon Allows multiple polygons to be treated as a single record
GeometryCollection Allows a single record to hold multiple geometry types

GeoPandas also provides spatial predicates, or logical tests that return True or False, which can be useful for conducting spatial queries. These tests are described in Table 9.2. Each method compares each geometry in a GeoSeries or GeoDataFrame to another geometry.

Table 9.2. GeoPandas spatial predicates.
Method Use
intersects() Do A and B intersect?
disjoint() Are A and B disjoint?
touches() Do A and B touch?
crosses() Do A and B cross?
within() Is A within B?
contains() Does A contain B?
overlaps() Do A and B overlap?
geom_equals() Are A and B identical?
covers() Does A cover B?
covered_by() Is A covered by B?
dwithin() Is A within a given distance of B?

Methods are also available for obtaining or transforming the datum/projection information for a layer. For example, the crs property provides the datum/projection information while the to_crs() method allows for converting between datums/projections. This can be useful when layers being used in an analysis do not have the same projection or coordinate reference system. If a layer is missing its coordinate reference system information, but you know what it should be, it can be defined using set_crs().

When performing overlay or spatial analysis involving multiple data layers using GeoPandas, the layers should use the same datum/projection information. If they do not, you can use to_crs() for datum/projection conversion.

There are also properties and methods available for calculating geometric properties, as described in Table 9.3. These calculations are made in the units of the layer’s coordinate reference system.

It is generally best to calculate geometric properties using a projected coordinate system as opposed to a geographic coordinate system (i.e., latitude and longitude). GeoPandas will warn you if you calculate areas or lengths using geographic coordinates.

Table 9.3. GeoPandas geometric calculations.
Property or Method Use
area Area of each polygon in the units of the coordinate reference system
length Length of each line, or perimeter of each polygon, in the units of the coordinate reference system
boundary Boundary of each geometry (e.g., the outline of a polygon as a line)
distance() Distance between each geometry and another geometry

9.2.2 Read Data

We first import the required libraries. Shapely, which is installed along with GeoPandas, is used to create new geometries. We use a variety of data layers, and all of the vector layers are stored in the vector.gpkg GeoPackage in the datasets/gisData/vector folder.

  • circle: a circle
  • triangle: a triangle
  • extent: bounding box extent of circle and triangle
  • interstates: interstates in West Virginia as lines
  • rivers: major rivers in West Virginia as lines
  • towns: point features of large towns in West Virginia
  • wv_counties: West Virginia county boundaries as polygons
  • harvest.csv: CSV table of deer harvest data by county from the West Virginia Division of Natural Resources (WVDNR)
  • tornadoesUS: point features of tornadoes across the US
  • citiesUS: point features of cities in the US
  • interstatesUS: interstates in the US as lines
  • us_eco_ref_l3US: ecological regions of the US as polygons
  • statesUS: US state boundaries

All vector layers are read in using the read_file() function from GeoPandas. This generates a GeoDataFrame. Tabulated data as a CSV file, harvest.csv, are read using read_csv() from Pandas.

When reading a layer from a database, such as an Esri File Geodatabase or a GeoPackage, the path to the database is provided first, and the name of the layer is provided to the layer parameter. If the file is not within a database, such as a shapefile or GeoJSON file, you only need to provide the path to the file. The list_layers() function lists the layers in a database along with their geometry types.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import geopandas as gpd
from shapely.geometry import box, Polygon

gpkg = "datasets/gisData/vector/vector.gpkg"
print(gpd.list_layers(gpkg))
               name    geometry_type
0            circle     MultiPolygon
1            extent     MultiPolygon
2             lines  MultiLineString
3            points            Point
4         rectangle     MultiPolygon
5          triangle     MultiPolygon
6       interstates  MultiLineString
7            rivers  MultiLineString
8             towns            Point
9       wv_counties     MultiPolygon
10         citiesUS            Point
11    interstatesUS  MultiLineString
12         statesUS     MultiPolygon
13      tornadoesUS            Point
14  us_eco_ref_l3US     MultiPolygon
circle = gpd.read_file(gpkg, layer="circle")
triangle = gpd.read_file(gpkg, layer="triangle")
ext = gpd.read_file(gpkg, layer="extent")
inter = gpd.read_file(gpkg, layer="interstates")
rivers = gpd.read_file(gpkg, layer="rivers")
towns = gpd.read_file(gpkg, layer="towns")
wv_counties = gpd.read_file(gpkg, layer="wv_counties")
harvest = pd.read_csv("datasets/gisData/vector/harvest.csv")
torn = gpd.read_file(gpkg, layer="tornadoesUS")
us_cities = gpd.read_file(gpkg, layer="citiesUS")
us_inter = gpd.read_file(gpkg, layer="interstatesUS")
eco = gpd.read_file(gpkg, layer="us_eco_ref_l3US")
states = gpd.read_file(gpkg, layer="statesUS")

Printing the first set of rows from the rivers layer, we can see that the data are formatted as a table where each row represents a spatial feature and the “geometry” column stores the geometric data. All other columns are attributes associated with the spatial features. The crs property shows that these data use the WGS84 UTM Zone 17N projection.

print(rivers.head())
       LENGTH  ...                                           geometry
0   1062.3540  ...  MULTILINESTRING ((707174.193 4377345.718, 7072...
1  10235.4600  ...  MULTILINESTRING ((708100.916 4373914.771, 7080...
2    144.6074  ...  MULTILINESTRING ((707964.385 4376732.736, 7080...
3  23608.8600  ...  MULTILINESTRING ((708085.323 4376702.738, 7086...
4   5451.6440  ...  MULTILINESTRING ((704804.496 4372629.738, 7051...

[5 rows x 6 columns]
print(type(rivers))
<class 'geopandas.geodataframe.GeoDataFrame'>
print(rivers.crs)
EPSG:32617

9.3 Data Wrangling and Query

Let’s start by mapping the ecological region boundaries using the plot() method of the GeoDataFrame, which is discussed in more detail in Chapter 8. We use the fill color to show the Level 1 ecoregion name as a qualitative/categorical variable by providing the column name to the column parameter. We also plot the legend outside of the map space.

legend_out = {"loc": "upper left", "bbox_to_anchor": (1, 1)}

fig, ax = plt.subplots(figsize=(10, 6))
eco.plot(ax=ax, column="NA_L1NAME", cmap="tab20", legend=True, legend_kwds=legend_out)
ax.set_axis_off()
plt.show()

Since GeoDataFrames act as DataFrames, we can work with and manipulate them using the Pandas methods discussed in Chapter 4. In the next code block, we count the number of polygons belonging to each Level 1 ecoregion using groupby() and size(). We can confirm that the Eastern Temperate Forests ecoregion has the largest number of polygons associated with it. This process is the same as working with any DataFrame, which is one of the benefits of using GeoPandas to work with vector data in Python. Since a summary statistic is calculated for a single column, the result is a Pandas Series as opposed to a GeoDataFrame. If you need to remove the geometric information from a GeoDataFrame, you can drop the “geometry” column using drop(columns="geometry"), which returns a Pandas DataFrame.

eco_l1_names = eco.groupby("NA_L1NAME").size().sort_values(ascending=False)
print(eco_l1_names)
NA_L1NAME
EASTERN TEMPERATE FORESTS          890
TROPICAL WET FORESTS                92
MARINE WEST COAST FOREST            75
GREAT PLAINS                        46
NORTHERN FORESTS                    43
MEDITERRANEAN CALIFORNIA            40
NORTHWESTERN FORESTED MOUNTAINS     38
NORTH AMERICAN DESERTS              13
TEMPERATE SIERRAS                   11
SOUTHERN SEMI-ARID HIGHLANDS         2
dtype: int64

Records, rows, or geospatial features can be extracted using a Boolean query. In the next code block, we extract all features that are within the Great Plains Level 1 ecoregion. We then map the extracted features and use the fill color to differentiate the Level 2 ecoregions. Only the Level 2 ecoregions that occur in the subset are included in the legend.

eco_gp = eco[eco["NA_L1NAME"] == "GREAT PLAINS"]

fig, ax = plt.subplots(figsize=(10, 6))
eco_gp.plot(ax=ax, column="NA_L2NAME", cmap="Set2", legend=True, legend_kwds=legend_out)
ax.set_axis_off()
plt.show()

Again, since GeoDataFrames behave as DataFrames, we can explore the tabulated data using a variety of methods. As an example, we use Seaborn (Chapter 5) to compare state-level population by sub-region of the US. Since the geographic information is not needed here, it is simply ignored.

fig, ax = plt.subplots(figsize=(10, 4))
sns.boxplot(data=states, x="SUB_REGION", y="POP2000", hue="SUB_REGION", ax=ax)
ax.set_xlabel("Sub-Region")
ax.set_ylabel("Population (2000)")
plt.show()

As a similar example, we now obtain the mean state population for each sub-region.

print(states.groupby("SUB_REGION")["POP2000"].mean())
SUB_REGION
E N Cen    9.031007e+06
E S Cen    4.255702e+06
Mid Atl    1.322395e+07
Mtn        2.271537e+06
N Eng      2.320420e+06
Pacific    1.439572e+07
S Atl      5.752129e+06
W N Cen    2.748248e+06
W S Cen    7.861212e+06
Name: POP2000, dtype: float64

Each tornado point has a Fujita scale attribute, a measure of tornado intensity. We now count the number of tornadoes by Fujita category. As expected, severe tornadoes occur much less frequently. We next extract all tornadoes that have a Fujita value greater than or equal to 3 and plot them as points over the state boundaries. As discussed in Chapter 8, multiple layers are displayed in the same map by plotting each to the same axes.

print(torn.groupby("F_SCALE").size())
F_SCALE
0    1236
1     531
2     168
3      58
4       6
5       1
dtype: int64
torn_hf = torn[torn["F_SCALE"] >= 3]

fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
torn_hf.plot(ax=ax, color="red", markersize=15)
ax.set_axis_off()
plt.show()

It is also possible to subset columns or attributes using a list of column names. Using this method, we extract only the state name, sub-region, and population attributes. In contrast to R’s sf package, the geometry column is not retained automatically. If the “geometry” column is not included in the list, the result is a Pandas DataFrame as opposed to a GeoDataFrame.

states_sel = states[["STATE_NAME", "SUB_REGION", "POP2000", "geometry"]]
print(type(states_sel))
<class 'geopandas.geodataframe.GeoDataFrame'>
states_sel.info()
<class 'geopandas.geodataframe.GeoDataFrame'>
RangeIndex: 49 entries, 0 to 48
Data columns (total 4 columns):
 #   Column      Non-Null Count  Dtype   
---  ------      --------------  -----   
 0   STATE_NAME  49 non-null     str     
 1   SUB_REGION  49 non-null     str     
 2   POP2000     49 non-null     int32   
 3   geometry    49 non-null     geometry
dtypes: geometry(1), int32(1), str(2)
memory usage: 2.2 KB

9.4 Table Joins

Tabulated data can be joined to spatial features if they share a common field or attribute. This is known as a table join. The goal of the next code block is to join the deer harvest tabulated data to the county boundaries. First, we change the “COUNTY” column’s name in the deer harvest table to “NAME” using rename() so that the column name matches that of the polygon features. We then use the merge() method to associate the data using the common “NAME” field. Setting how="inner" performs an inner join, in which only records that occur in both tables are retained. Since the merge() method is applied to the GeoDataFrame, the result is a GeoDataFrame. Lastly, we map the total number of deer harvested by county.

harvest = harvest.rename(columns={"COUNTY": "NAME"})

counties_harvest = wv_counties.merge(harvest, on="NAME", how="inner")

fig, ax = plt.subplots(figsize=(8, 6))
counties_harvest.plot(ax=ax, column="TOTALDEER", cmap="Reds", edgecolor="grey", legend=True,
                      legend_kwds={"label": "Harvest", "shrink": 0.7})
ax.set_axis_off()
plt.show()

New attribute columns can be generated using basic column math. We calculate the density of deer harvested as deer per square mile using the “TOTALDEER” and “SQUARE_MIL” fields. The density is written to a new column called “DEN”, and the result is mapped. Next, we extract and map all counties that have a deer harvest density greater than five deer per square mile.

counties_harvest["DEN"] = counties_harvest["TOTALDEER"] / counties_harvest["SQUARE_MIL"]

fig, ax = plt.subplots(figsize=(8, 6))
counties_harvest.plot(ax=ax, column="DEN", cmap="Reds", edgecolor="grey", legend=True,
                      legend_kwds={"label": "Harvest Density", "shrink": 0.7})
ax.set_axis_off()
plt.show()

counties_harvest_sub = counties_harvest[counties_harvest["DEN"] > 5]

fig, ax = plt.subplots(figsize=(8, 6))
counties_harvest.plot(ax=ax, color="lightgrey", edgecolor="grey")
counties_harvest_sub.plot(ax=ax, column="DEN", cmap="Reds", edgecolor="grey", legend=True,
                          legend_kwds={"label": "Harvest Density", "shrink": 0.7})
ax.set_axis_off()
plt.show()

The area property returns the area of each polygon feature in the map units of the coordinate reference system. In our example, the units are square meters since the data are projected to a UTM coordinate system. In the following code block, we calculate area in hectares and write the values to a new column called “hec”. We then calculate density again using hectares as opposed to square miles.

counties_harvest["hec"] = counties_harvest.area * 0.0001
counties_harvest["DEN2"] = counties_harvest["TOTALDEER"] / counties_harvest["hec"]

fig, ax = plt.subplots(figsize=(8, 6))
counties_harvest.plot(ax=ax, column="DEN2", cmap="Reds", edgecolor="grey", legend=True,
                      legend_kwds={"label": "Harvest Density (deer/ha)", "shrink": 0.7})
ax.set_axis_off()
plt.show()

9.5 Geometry Type Conversions

9.5.1 Bounding Boxes and Centroids

The total_bounds property returns the bounding box coordinates (minimum x, minimum y, maximum x, maximum y) for an entire layer, while the bounds property returns the bounding box of each feature. These do not create spatial features. To convert the bounding box coordinates to a polygon, we use the box() function from Shapely. The * unpacks the four coordinates into separate arguments. We then convert the result to a GeoSeries with the same coordinate reference system as the counties. In the map, the bounding box for West Virginia is shown in red. The envelope property, which is not demonstrated, returns the bounding box of each feature as a polygon.

bb_wv = gpd.GeoSeries([box(*wv_counties.total_bounds)], crs=wv_counties.crs)

fig, ax = plt.subplots(figsize=(8, 6))
wv_counties.plot(ax=ax, color="tan", edgecolor="grey")
bb_wv.boundary.plot(ax=ax, color="red", linewidth=2)
ax.set_axis_off()
plt.show()

The centroid property returns the centroid of each polygon feature as a Point geometry. The centroid does not always occur inside of the polygon that it is the center of (for example, a “C-shaped” island or a doughnut). The representative_point() method provides an alternative in which a point within the polygon is returned. Note that centroid returns a GeoSeries of geometries only. To keep the attributes, we use set_geometry() to replace the polygon geometry of the counties with the centroids. As demonstrated by printing the column names, the attributes are retained.

wv_centroids = wv_counties.set_geometry(wv_counties.centroid)

fig, ax = plt.subplots(figsize=(8, 6))
wv_counties.plot(ax=ax, color="tan", edgecolor="grey")
wv_centroids.plot(ax=ax, color="red", markersize=30)
ax.set_axis_off()
plt.show()

print(wv_centroids.columns.tolist())
['AREA', 'PERIMETER', 'NAME', 'STATE', 'FIPS', 'STATE_ABBR', 'SQUARE_MIL', 'POP2000', 'POP00SQMIL', 'MALE2000', 'FEMALE2000', 'MAL2FEM', 'UNDER18', 'AIAN', 'ASIA', 'BLACK', 'NHPI', 'WHITE', 'AIAN_MORE', 'ASIA_MORE', 'BLK_MORE', 'NHPI_MORE', 'WHT_MORE', 'HISP_LAT', 'F1900', 'F1910', 'F1920', 'F1930', 'F1940', 'F1950', 'F1960', 'F1970', 'F1980', 'F1991', 'F1992', 'F1993', 'F1994', 'F1995', 'F1996', 'F1997', 'F1998', 'F1999', 'F1990', 'F2000', 'F2001', 'F2002', 'F2003', 'F2004', 'F2005', 'F2006', 'geometry']

9.5.2 Thiessen/Voronoi Polygons

Thiessen or Voronoi polygons represent the areas that are closer to one point feature than any other feature in a dataset. The voronoi_polygons() method generates these polygons. Since the diagram is calculated from the full set of points, we first merge all centroids into a single MultiPoint geometry using union_all() and convert it back to a GeoSeries. The result is a single GeometryCollection, which is converted to individual polygons using explode().

We then extract the polygon extents within the state boundary using the clip() method. The state boundary is generated by dissolving the counties, which is explained later in this chapter. In short, this process serves as a means to convert points to polygons.

vor_poly = (gpd.GeoSeries([wv_centroids.union_all()], crs=wv_counties.crs)
            .voronoi_polygons().explode(index_parts=False))

wv_diss = wv_counties.dissolve(by="STATE", aggfunc={"POP2000": "sum"})

vor_poly2 = vor_poly.clip(wv_diss)

fig, ax = plt.subplots(figsize=(8, 6))
wv_diss.plot(ax=ax, color="tan")
vor_poly2.boundary.plot(ax=ax, color="red", linewidth=2)
wv_centroids.plot(ax=ax, color="blue", markersize=20)
ax.set_axis_off()
plt.show()

9.5.3 Convex Hull

A convex hull is a bounding geometry that encompasses features without any concave angles. Think of this as stretching a rubber band around the margin points in a dataset. A convex hull can be generated using the convex_hull property. Similar to the Voronoi polygons, the Point geometries must first be merged to a MultiPoint geometry. It is also possible to generate a concave hull using the concave_hull() method, where the ratio parameter controls the degree of concavity.

ch = gpd.GeoSeries([wv_centroids.union_all().convex_hull], crs=wv_counties.crs)

fig, ax = plt.subplots(figsize=(8, 6))
wv_counties.plot(ax=ax, color="tan", edgecolor="grey")
wv_centroids.plot(ax=ax, color="red", markersize=30)
ch.boundary.plot(ax=ax, color="blue", linewidth=2)
ax.set_axis_off()
plt.show()

9.6 Spatial Joins

In contrast to a table join, a spatial join associates features based on spatial co-occurrence as opposed to a shared or common attribute. We use the sjoin() function to spatially join the cities and states. The predicate parameter defines the spatial relationship; here, each city is joined to the state that it is within. The result is a Point geometry with each city containing its original attributes plus the attributes for the state in which it occurs. An “index_right” column is also added, which stores the index of the matching state. To demonstrate this, we symbolize the points using the “SUB_REGION” field, which was initially attributed to the states layer. Since both layers include a “POP2000” column, the overlapping column names are given suffixes (“POP2000_left” and “POP2000_right”).

cities2 = gpd.sjoin(us_cities, states, how="left", predicate="within")

fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
cities2.plot(ax=ax, column="SUB_REGION", cmap="tab10", markersize=8, legend=True,
             legend_kwds=legend_out)
ax.set_axis_off()
plt.show()

9.7 Random Sampling using the Table

Randomly sampling spatial features is the same as randomly sampling records or rows from any DataFrame. In the first example, we demonstrate simple random sampling without replacement using the sample() method. By default, features are not selected multiple times. The result is 100 randomly selected cities. Since this is random, we will obtain a different result each time we run the code unless we set the random seed using the random_state parameter. It is also possible to stratify the sampling using groupby(). In the second example, we select 10 random cities per sub-region. To confirm that the desired result was obtained, we next count the selected cities by sub-region.

cities_rswor = cities2.sample(n=100)

fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
cities2.plot(ax=ax, color="black", markersize=3)
cities_rswor.plot(ax=ax, color="red", markersize=20)
ax.set_axis_off()
plt.show()

cities_srswor = cities2.groupby("SUB_REGION").sample(n=10, random_state=42)

print(cities_srswor.groupby("SUB_REGION").size())
SUB_REGION
E N Cen    10
E S Cen    10
Mid Atl    10
Mtn        10
N Eng      10
Pacific    10
S Atl      10
W N Cen    10
W S Cen    10
dtype: int64
fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
cities_srswor.plot(ax=ax, column="SUB_REGION", cmap="tab10", markersize=20, legend=True,
                   legend_kwds=legend_out)
ax.set_axis_off()
plt.show()

9.8 Geoprocessing

9.8.1 Boundary Generalization (Dissolve)

Dissolve is used for boundary generalization. For example, you can dissolve county boundaries to state boundaries or state boundaries to the country boundary. In the example, we obtain sub-region boundaries from the state boundaries using the dissolve() method. The by parameter defines the attribute used to group the features. If you would like to include a summary of some attributes relative to the dissolved features, you can provide a dictionary to the aggfunc parameter. Below, we sum the population of all the states in each sub-region and rename the result to “totpop”. In the second example, we generate the West Virginia state boundary from the county boundaries. Since all counties have the same value in the “STATE” field, a single feature is returned. We have also summed the population of the counties, which results in the total state population. Note that the grouping attribute becomes the index of the result.

sub_regions = (states.dissolve(by="SUB_REGION", aggfunc={"POP2000": "sum"})
               .rename(columns={"POP2000": "totpop"}))
print(sub_regions.drop(columns="geometry"))
              totpop
SUB_REGION          
E N Cen     45155037
E S Cen     17022810
Mid Atl     39671861
Mtn         18172295
N Eng       13922517
Pacific     43187168
S Atl       51769160
W N Cen     19237739
W S Cen     31444850
fig, ax = plt.subplots(figsize=(10, 6))
sub_regions.reset_index().plot(ax=ax, column="SUB_REGION", cmap="tab10", legend=True,
                               legend_kwds=legend_out)
ax.set_axis_off()
plt.show()

wv_diss = (wv_counties.dissolve(by="STATE", aggfunc={"POP2000": "sum"})
           .rename(columns={"POP2000": "totpop"}))
print(wv_diss.drop(columns="geometry"))
        totpop
STATE         
54     1808344
fig, ax = plt.subplots(figsize=(8, 6))
wv_diss.plot(ax=ax, color="tan")
ax.set_axis_off()
plt.show()

9.8.2 Proximity Analysis (Buffer)

Proximity analysis relates to finding features that are near to other features. A buffer represents the area that is within a defined distance of an input feature. Buffers can be generated for Points, LineStrings, or Polygons. They can also be generated for the multipart versions of these geometries. The result will always be a polygon or area. Below, we create polygon buffers that encompass all areas within 20 km of the mapped interstates using the buffer() method. The buffer distance is defined in the map units, in this case meters. Since buffer() returns a GeoSeries, we convert the result to a GeoDataFrame.

us_inter_20km = gpd.GeoDataFrame(geometry=us_inter.buffer(20000), crs=us_inter.crs)

fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
us_inter_20km.plot(ax=ax, color="red", alpha=0.6)
us_inter.plot(ax=ax, color="black", linewidth=0.5)
ax.set_axis_off()
plt.show()

9.8.3 Vector Overlay

9.8.3.1 Intersect/Clip (AND)

Clip and intersect are used to extract features within another feature or return the overlapping extent of multiple features. Think of this as a spatial AND. The overlay() function performs vector overlay; the type of overlay is defined using the how parameter. When we intersect the circle and triangle, the result is the area that occurs in both the circle and the triangle (shown in red). All attributes from both features are returned, as demonstrated by printing the output. The table contains both the “C” and “T” fields, which came from the circle and triangle, respectively. An intersection is equivalent to the Intersect Tool in ArcGIS Pro.

The overlay() function with how="intersection" and the intersects() method are sometimes confused. The former performs a spatial intersection to return a new geometry while the latter is a test that returns True or False. It specifically tests whether two geometries intersect.

def overlay_map(result, show_inputs=True):
    """Map an overlay result (red) relative to the circle and triangle."""
    fig, ax = plt.subplots(figsize=(6, 6))
    ext.boundary.plot(ax=ax, color="black")
    if show_inputs:
        circle.plot(ax=ax, color="#0D5C86")
        triangle.plot(ax=ax, color="#67B38C")
    result.plot(ax=ax, color="#FD694F", edgecolor="black")
    ax.set_axis_off()
    plt.show()

isect_example = gpd.overlay(circle, triangle, how="intersection")
overlay_map(isect_example)

print(isect_example)
   Id_1  C  ...         area                                           geometry
0     0  C  ...  7360.590638  POLYGON ((587683.915 4395515.173, 587944.053 4...

[1 rows x 6 columns]

In this second example, we find all tornadoes that occurred within 20 km of an interstate. When the features being extracted are points, the clip() function is a convenient option: it returns the features, or portions of features, within the extent of the polygons. So, buffer and clip can be combined to extract features within a specified distance of other features.

torn_near = gpd.clip(torn, us_inter_20km)
print(len(torn), len(torn_near))
2000 872
fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
us_inter_20km.plot(ax=ax, color="red", alpha=0.6)
us_inter.plot(ax=ax, color="black", linewidth=0.5)
torn_near.plot(ax=ax, color="black", markersize=5)
ax.set_axis_off()
plt.show()

9.8.3.2 Union (OR)

Using how="union" with overlay() is equivalent to the Union Tool in ArcGIS Pro: the full extent of both layers is returned, and the internal boundaries are maintained, so the result contains the parts of the circle and triangle that do and do not overlap as separate features. If you would like to merge the features into a single feature with no internal boundaries, the result can be dissolved. Since no by argument is provided to dissolve(), all features are merged into one. Think of this as a spatial OR.

union_example = gpd.overlay(circle, triangle, how="union")
print(union_example)
   Id_1    C  ...         area                                           geometry
0   0.0    C  ...  7360.590638  POLYGON ((587683.915 4395515.173, 587944.053 4...
1   0.0    C  ...          NaN  POLYGON ((658593.562 4313952.621, 658607.352 4...
2   NaN  NaN  ...  7360.590638  POLYGON ((645177.84 4498574.85, 750422.638 431...

[3 rows x 6 columns]
union_diss = union_example.dissolve()
overlay_map(union_diss, show_inputs=False)

9.8.3.3 Difference (NOT)

Using how="difference" returns the portion of the first layer that is not in the second. This is similar to the Erase Tool in ArcGIS Pro. This is a spatial NOT: circle NOT triangle. The result is the portion of the circle shown in red in the map below.

erase_example = gpd.overlay(circle, triangle, how="difference")
overlay_map(erase_example)

9.8.3.4 Symmetrical Difference (XOR)

Symmetrical difference is a spatial XOR: circle OR triangle, excluding circle AND triangle. So, the result is the area in either the circle or the triangle, but not the area in both.

sym_diff_examp = gpd.overlay(circle, triangle, how="symmetric_difference")
overlay_map(sym_diff_examp, show_inputs=False)

9.9 Data Summarization

9.9.1 Count Points in Polygon

You can count the number of point features occurring in polygons using GeoPandas and Pandas. The goal of the code block below is to count the number of cities in each state. This is accomplished using the following process.

  1. Use sjoin() to join the cities and states based on spatial co-occurrence; this results in Point geometries where each point has its original attributes plus the attributes for the state in which it occurs.
  2. Use groupby() and size() to count the number of cities by state; the result is a Pandas Series with no geometric information.
  3. Join the counts back to the state boundaries using the common “STATE_NAME” field and a table join (merge()). A left join is used so that all states are retained.

Once this process is complete, we then map the resulting counts.

cities_in_state = gpd.sjoin(us_cities, states, how="inner", predicate="within")

cities_cnt = cities_in_state.groupby("STATE_NAME").size().rename("n").reset_index()

state_cnt = states.merge(cities_cnt, on="STATE_NAME", how="left")

fig, ax = plt.subplots(figsize=(10, 6))
state_cnt.plot(ax=ax, column="n", cmap="Oranges", edgecolor="grey", linewidth=0.5,
               legend=True, legend_kwds={"label": "Number of Cities", "shrink": 0.6})
ax.set_axis_off()
plt.show()

9.9.2 Length of Lines in Polygon

You can also sum the length of lines within polygons. The goal of the code below is to sum the length of interstates by state using the following process:

  1. Intersect the interstates and states using overlay() to obtain LineString geometries where the interstates have been split along state boundaries and now have the attributes for the state in which they occur. Setting keep_geom_type=True returns only line geometries.
  2. Use the length property to calculate the length of each line segment. It is not appropriate to use the original length field because it does not reflect the length if the interstate was split along a state border. We divide by 1,000 to obtain the length in kilometers as opposed to meters.
  3. Group the lines by state and sum the length using groupby() and sum().
  4. Join the result back to the state boundaries using merge() and the common “STATE_NAME” field.
us_inter_isect = gpd.overlay(us_inter, states, how="intersection", keep_geom_type=True)
us_inter_isect["ln_km"] = us_inter_isect.length / 1000

state_ln = us_inter_isect.groupby("STATE_NAME")["ln_km"].sum().rename("inter_ln").reset_index()

state_inter_ln = states.merge(state_ln, on="STATE_NAME", how="left")

fig, ax = plt.subplots(figsize=(10, 6))
state_inter_ln.plot(ax=ax, column="inter_ln", cmap="Purples", edgecolor="grey", linewidth=0.5,
                    legend=True, legend_kwds={"label": "Interstate Length (km)", "shrink": 0.6})
ax.set_axis_off()
plt.show()

For a fairer comparison, it would make sense to calculate the density of interstates as the length of interstates in kilometers per square kilometer of state area. Again, this can be accomplished using only GeoPandas and Pandas. Here, we display the result using a quantile classification with five classes using the scheme and k parameters, which are discussed in Chapter 8.

state_inter_ln["sq_km"] = state_inter_ln.area * 1e-6
state_inter_ln["int_den"] = state_inter_ln["inter_ln"] / state_inter_ln["sq_km"]

fig, ax = plt.subplots(figsize=(10, 6))
state_inter_ln.plot(ax=ax, column="int_den", cmap="Purples", scheme="quantiles", k=5,
                    edgecolor="grey", linewidth=0.5, legend=True,
                    legend_kwds={"title": "km per square km", **legend_out})
ax.set_axis_off()
plt.show()

9.10 Other Operations

9.10.1 Simplify Geometry

The simplify() method is used to simplify or generalize features. Higher tolerance values result in more simplification. The tolerance is defined in map units, in this case meters. Topology can be preserved using the preserve_topology parameter. For simplifying the state boundaries, topology is preserved in the first example below but not the second. Without preserving topology, the smallest states collapse to empty geometries and disappear from the map; the is_empty property indicates that this occurs for five states. In the third example, a simplification of the West Virginia state boundary is generated using a tolerance of 10,000 meters with topology preserved.

def simple_map(gdf):
    fig, ax = plt.subplots(figsize=(10, 6))
    gdf.plot(ax=ax, color="tan", edgecolor="black", linewidth=0.5)
    ax.set_axis_off()
    plt.show()

state_100000 = states.simplify(100000, preserve_topology=True)
simple_map(state_100000)

state_100000 = states.simplify(100000, preserve_topology=False)
print(state_100000.is_empty.sum())
5
simple_map(state_100000)

wv_bound = states[states["STATE_NAME"] == "West Virginia"].simplify(10000, preserve_topology=True)
simple_map(wv_bound)

9.10.2 Spatial Sampling

Random points can be generated within polygons using the sample_points() method. In the example, we generate 400 random points across the country. Since sample_points() generates points within each feature, we first merge the states into a single polygon using union_all(). The result is a single MultiPoint geometry, which is converted to individual points using explode(). Since this is random, a different result is obtained if the code is re-executed unless a random seed is set using the rng parameter. Applying sample_points() to the states without merging them results in a stratified sample, since points are generated separately within each state. In the second example, we generate 10 random points in each state.

us_poly = gpd.GeoSeries([states.union_all()], crs=states.crs)
pnt_sample = us_poly.sample_points(400).explode(index_parts=False)

fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
pnt_sample.plot(ax=ax, color="black", markersize=5)
ax.set_axis_off()
plt.show()

pnt_sample2 = states.sample_points(10, rng=42).explode(index_parts=False)
print(len(pnt_sample2))
490
fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
pnt_sample2.plot(ax=ax, color="black", markersize=5)
ax.set_axis_off()
plt.show()

9.10.3 Tessellation

GeoPandas does not provide a function for generating a tessellation, but one is easy to build using Shapely and NumPy. Below, we write a function that generates a rectangular grid tessellation over an extent. The linspace() function from NumPy defines the x and y coordinates of the cell edges, and box() creates a polygon for each cell. We then create a grid of 10-by-10 cells within the extent of the states.

def make_grid(bounds, nx, ny, crs):
    """Create a rectangular grid of nx by ny cells covering the bounds."""
    xmin, ymin, xmax, ymax = bounds
    xs = np.linspace(xmin, xmax, nx + 1)
    ys = np.linspace(ymin, ymax, ny + 1)
    cells = [box(x0, y0, x1, y1)
             for x0, x1 in zip(xs[:-1], xs[1:]) for y0, y1 in zip(ys[:-1], ys[1:])]
    return gpd.GeoDataFrame(geometry=cells, crs=crs)

us_grid = make_grid(states.total_bounds, 10, 10, states.crs)

fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
us_grid.plot(ax=ax, facecolor=(1, 0, 0, 0.2), edgecolor="black", linewidth=2)
ax.set_axis_off()
plt.show()

A hexagonal tessellation can be generated in a similar manner. The function below creates flat-topped hexagons by calculating the six vertices of each hexagon from its center point. The hexagon centers are offset by half a hexagon height in every other column so that the hexagons fit together without gaps.

def make_hex_grid(bounds, n, crs):
    """Create a grid of flat-topped hexagons with about n columns covering the bounds."""
    xmin, ymin, xmax, ymax = bounds
    size = (xmax - xmin) / (1.5 * n)       # distance from hexagon center to vertex
    width, height = 1.5 * size, np.sqrt(3) * size
    angles = np.radians(np.arange(0, 360, 60))
    hexes = []
    for i, x in enumerate(np.arange(xmin, xmax + width, width)):
        offset = height / 2 if i % 2 else 0
        for y in np.arange(ymin - offset, ymax + height, height):
            hexes.append(Polygon(zip(x + size * np.cos(angles), y + size * np.sin(angles))))
    return gpd.GeoDataFrame(geometry=hexes, crs=crs)

us_hex = make_hex_grid(states.total_bounds, 10, states.crs)

fig, ax = plt.subplots(figsize=(10, 6))
states.plot(ax=ax, color="tan", edgecolor="grey", linewidth=0.5)
us_hex.plot(ax=ax, facecolor=(1, 0, 0, 0.2), edgecolor="black", linewidth=2)
ax.set_axis_off()
plt.show()

9.10.4 Writing Data

Lastly, GeoDataFrames can be written to disk using the to_file() method. The output format is determined by the file extension, or it can be set using the driver parameter. When writing to a GeoPackage, the layer parameter defines the name of the layer. Below, we save the counties with the deer harvest data to a GeoPackage and the cities with the state attributes to a GeoJSON file. Since a GeoPackage can store many layers, it is a convenient format for organizing the data for a project.

counties_harvest.to_file("output/vector_results.gpkg", layer="counties_harvest", driver="GPKG")
cities2.drop(columns="index_right").to_file("output/cities_states.geojson", driver="GeoJSON")
print(gpd.list_layers("output/vector_results.gpkg"))
               name geometry_type
0  counties_harvest  MultiPolygon

9.11 Concluding Remarks

We have tried to focus on common geoprocessing and vector analysis techniques here. You will likely run into specific issues that will require different techniques. However, we think you will find that the examples provided offer a starting point for other analyses. In the next chapter, we explore working with raster grids.

9.12 Questions

  1. Explain the difference between Polygon and MultiPolygon geometries.
  2. Explain the concept of a GeometryCollection. How is this different from other geometries?
  3. Explain the difference between a table join, such as those implemented with the merge() method, and a spatial join implemented with sjoin().
  4. Explain the purpose of the layer parameter of read_file(). When is it needed?
  5. Explain the difference between using overlay() with how="intersection" and using the intersects() method.
  6. Explain the differences between the "intersection", "union", "difference", and "symmetric_difference" options for the how parameter of overlay().
  7. Within the simplify() method, what is the purpose of the preserve_topology parameter?
  8. Why does selecting a subset of columns from a GeoDataFrame sometimes return a Pandas DataFrame as opposed to a GeoDataFrame?
  9. Explain how you might calculate the area of polygons relative to other polygons (e.g., the percent of each county covered by public land) using GeoPandas. This was not demonstrated in our examples.

9.13 Exercises

Exercise 9.1: Vector Analysis of West Virginia Data

Introduction

The goal of this exercise is to work with and analyze vector data in Python using a wide variety of vector querying, analysis, and overlay methods and the GeoPandas library, as demonstrated in Chapter 9. The data layers are provided in the vector_exercise.gpkg GeoPackage in the datasets/gisData/vector/exercise folder and were obtained from the West Virginia GIS Technical Center.

  • springs: point locations of springs in West Virginia
  • towns: point locations of towns in West Virginia
  • interstates: line features of interstates in West Virginia
  • major_rivers: line features of major rivers in West Virginia
  • counties: polygon features of West Virginia county boundaries
  • geology: polygon features of geologic formations in West Virginia
Deliverables
  • Jupyter Notebook or Quarto document with all code and maps embedded. Answers to questions should be provided in Markdown cells or text.

Questions and Tasks

Use these data layers, Pandas, and GeoPandas to perform the following analyses and answer the associated questions.

  1. Read in all of the required layers using read_file().
  2. Make a map that includes the county boundaries, town points, interstates, and rivers to visualize the data.
  3. What percentage of towns have a population (“POPULATION”) greater than 1,500?
  4. How many towns are categorized as a village (use the “TYPE” field)?
  5. Randomly select 25 towns from the dataset.
  6. Randomly select three towns from each type (use the “TYPE” field).
  7. What percentage of springs have an average flow (“AVEFLOW”) greater than 700?
  8. What percentage of springs occur on terraces (use the “TOPO_POS” field)?
  9. Extract Interstate 79 from the interstate line features (use the “SIGN1” field).
  10. Dissolve the I-79 features to a single line feature and obtain the total length in kilometers (use the “KM” field, which provides the length in kilometers). What is the total length?
  11. Extract all towns within 10 km of I-79. How many towns are within 10 km of I-79?
  12. Create a map to visualize the results from the analysis that includes the county boundaries, I-79 line, I-79 buffer, and the extracted town points.
  13. Extract all springs in limestone geology (use the “TYPE” field in the geology layer). What percent of all springs occur in limestone?
  14. Create a table of counts of springs by rock type. Which rock type has the largest number of springs?
  15. Create 5 random points per county in the state.
  16. Create a map to visualize the randomly sampled points that includes the county boundaries and sampled points.
  17. Calculate the length of major rivers by county. Which county has the longest length of major rivers? (Note that the major_rivers layer does not include a length field, so you will need to calculate the lengths.)
  18. Create a map to visualize the length of rivers by county using color.
  19. Calculate the density of towns by county. Which county has the largest density of towns? (Note: both layers have a “NAME” field, so you will need to rename one of these fields to eliminate the overlap and confusion.)
  20. Create a map to visualize the density of towns by county using color.