Real-Time Spatial Data: Geofences, Windows and Late Events

A city bus company wants to know how many of its buses are inside the central zone every minute. The GPS units send a position every few seconds, over mobile networks that drop and delay messages. Counting buses sounds simple until you ask which position counts when two arrive in the wrong order.

This post builds a very small version of that system in Python, to show the three ideas behind most real-time spatial work.

What is real-time spatial data processing?

It means handling location data as it arrives, in seconds, instead of loading a file once and analysing it later. The input is a stream of events. An event here is one record: a vehicle ID, a position and a timestamp.

Two timestamps matter and they are not the same.

• Event time is when the position was recorded by the device.
• Arrival time is when your system received it.

Network delay means events can arrive out of order, so the order you receive them in is not the order they happened.

Why it matters

Live systems feed decisions that cannot wait for a nightly job: dispatching the nearest ambulance, alerting when a truck leaves its route, warning people when a sensor shows rising water. They also fail in quiet ways. A system that counts by arrival time instead of event time will report numbers that look reasonable and are wrong.

How it works

Three building blocks cover most cases.

1. Geofence check. Is this point inside this polygon? This is a point-in-polygon test. For a small number of zones it is cheap. For thousands of zones, use a spatial index (an R-tree, or PostGIS with a GiST index) so you test only nearby shapes.
2. Windowed aggregation. Group events into fixed time slices, such as one minute, and compute a value per slice. A fixed, non-overlapping slice is called a tumbling window.
3. Watermark and allowed lateness. A watermark is the system's statement that "I believe I have seen everything up to this time". You set a lateness allowance, such as 30 seconds. Events older than the watermark are treated as too late and handled separately, so a window can be closed and reported.

One CRS note: a point-in-polygon test with longitude and latitude (EPSG:4326) is fine, because it checks position and does not measure distance. If you add a distance rule such as "within 200 m of the depot", project the data first.

Practical GIS example

The script below simulates the bus example with a handful of hand-written events, including two that arrive out of order. It needs only Shapely.

# Count buses inside a zone per one-minute window, handling late events.
# pip install shapely

from collections import defaultdict
from shapely.geometry import Point, Polygon

# Step 1: The zone (longitude, latitude pairs) and the rules
zone = Polygon([(77.0, 28.5), (77.3, 28.5), (77.3, 28.8), (77.0, 28.8)])
WINDOW_SECONDS = 60
ALLOWED_LATENESS = 30

# Step 2: Events in the order they ARRIVE, not the order they happened.
# "time" is event time in seconds since the start.
events = [
    {"bus": "bus1", "time": 5,  "lon": 77.10, "lat": 28.60},
    {"bus": "bus2", "time": 20, "lon": 77.20, "lat": 28.70},
    {"bus": "bus1", "time": 65, "lon": 77.10, "lat": 28.60},
    {"bus": "bus3", "time": 50, "lon": 77.15, "lat": 28.65},  # arrives late
    {"bus": "bus2", "time": 10, "lon": 77.20, "lat": 28.70},  # arrives too late
    {"bus": "bus4", "time": 70, "lon": 78.00, "lat": 29.50},  # outside the zone
]

windows = defaultdict(set)   # window number -> set of buses seen inside
too_late = []
latest_event_time = 0

for event in events:
    # Step 3: Move the watermark forward using the newest event time seen
    latest_event_time = max(latest_event_time, event["time"])
    watermark = latest_event_time - ALLOWED_LATENESS

    # Step 4: Set aside events older than the watermark
    if event["time"] < watermark:
        too_late.append(event)
        continue

    # Step 5: Geofence check
    if zone.contains(Point(event["lon"], event["lat"])):
        # Step 6: Assign to a window by EVENT time
        window = event["time"] // WINDOW_SECONDS
        windows[window].add(event["bus"])

# Step 7: Report
for window in sorted(windows):
    print(f"Window {window}: {len(windows[window])} bus(es) in zone")
print("Too late to count:", [e["bus"] for e in too_late])

The output is:

Window 0: 3 bus(es) in zone
Window 1: 1 bus(es) in zone
Too late to count: ['bus2']

Walk through why. bus3 has event time 50 but arrives after an event at time 65, so the watermark is 35 and 50 is still inside the allowance. It goes into window 0 where it belongs. The second bus2 event, at time 10, arrives when the watermark is 35, so it is set aside. bus4 is processed but is outside the zone.

The example is simplified: it counts at the end, whereas a real system closes each window once the watermark passes it and then emits the result.

Workflow

For a real system the steps are the same, with different parts around them.

1. Receive events from a message broker such as Kafka or MQTT, or an HTTP endpoint.
2. Validate each one: coordinates in range, timestamp plausible, device known.
3. Apply the geofence or spatial join.
4. Aggregate by event-time window.
5. Write the live result to a fast store for the map, and write every raw event to cheaper storage for history.

Keeping a hot path for recent results and a cold path for the full history means the live map stays fast and you can still re-run the past when a rule changes.

Common mistakes

• Using arrival time instead of event time. Counts shift when the network is slow.
• Assuming events arrive in order. They do not, and your design needs a stated lateness rule.
• Trusting every GPS fix. Check for points at 0, 0, impossible speeds and jumps across the map.
• Testing every point against every polygon. It is fine at ten zones and too slow at ten thousand. Use a spatial index.
• Measuring in degrees. Project the data before any distance or buffer rule.
• Throwing away late events silently. Log or store them, so you know how much you are missing.

Key takeaways

• A real-time spatial system is a stream of timestamped events, not a dataset.
• Event time and arrival time are different, and windows should use event time.
• A watermark with an allowed lateness lets you close windows and still handle stragglers.
• Point-in-polygon checks work in latitude and longitude; distance checks need a projected CRS.
• Separate the hot path (live) from the cold path (history).