🌎 Helping geospatial professionals grow using technology · Scaling geospatial at Wherobots
Most people lump PostGIS, DuckDB, and Apache Sedona into the same mental bucket. They’re not. And if you don’t understand the difference, you’re either over engineering your workflows or breaking things at scale. Here’s the breakdown: 🐘 OTAP (Operational Transactional Analytic Processing) → PostGIS What it’s good at: Small- to mid-scale operational workloads. Think: a city planning department running spatial queries or updates against parcels, buffers, or intersections. It shines when you need transactional consistency, live updates, and integration with operational apps. What it’s not good at: PostGIS is not your friend when you dump 500M records of building footprints or retrieve a country-size batch of road data. You’ll end up with timeouts and an angry DBA. Running live queries to support a permitting system in a mid-sized city → ✅ Querying global-scale datasets on PostGIS → ❌ 📈OLAP (Online Analytical Processing) → DuckDB, Snowflake, BigQuery, Redshift What it’s good at: Analytical slicing and dicing of large but mostly tabular datasets. Think: millions (or billions) of rows of geotagged events, aggregated by region, month, or customer. They’re optimized for scans, filters, group-bys. Pair with formats like GeoParquet and you’ve got fast analytics. What it’s not good at: OLAP engines aren't the best at heavy processing. Even clipping NYC sidewalks by buffers around trees fell flat. They don’t handle complex geometries, raster data, or more complex spatial problems. You might hack around it with extensions or UDFs, but that’s like bringing a butter knife to cut down a tree. Example: Calculating average delivery times by ZIP code across billions of trips → ✅ Running viewshed analysis or cost distance models in an OLAP → ❌ 🏭 Processing Engines → Databricks, Wherobots, Sedona, Dask What they’re good at: Heavy lifting. These are built for ETL pipelines, spatial joins across hundreds of millions of features, distributed raster analysis, machine learning, and crunching petabytes of data. This is the layer where you do the big lift before pushing summaries downstream. What they’re not good at: You don’t spin up Spark to answer “how many parcels intersect this buffer?” Also, they’re not your transactional system: no one is building a permit approval workflow directly on Spark. Example: Intersecting 780M building footprints with global flood rasters → ✅ Checking whether a bike rack falls inside a single polygon → ❌ 👉 Don't do everything in one tool. The modern spatial stack is about using the right tool at the right layer so you don’t waste time (or money) fighting against the wrong one. 🌎 I'm Matt and I talk about modern geospatial. 📬 Want more like this? Join 8k+ others learning from my newsletter → forrest.nyc