GIS work starts and ends with files: a CSV of coordinates comes in, a cleaned file goes out. JSON is the format web services and GeoJSON use. This part covers the standard library tools for all of them, so that Pandas will feel familiar later.
What is a file path?
A path is the address of a file. On Windows it looks like C:\Users\you\data\cities.csv, and on macOS or Linux /home/you/data/cities.csv. Hard coding such paths makes a script work on one machine only. The pathlib module builds paths in a way that works anywhere.
1. Paths with pathlib
from pathlib import Path # Path handles file and folder names for you, on Windows, Mac and Linux
# Make a folder called gis_demo next to this notebook (exist_ok=True: no error if it already exists)
folder = Path("gis_demo")
folder.mkdir(exist_ok=True)
# The / joins a folder and a file name into one path
csv_path = folder / "cities.csv"
print(csv_path.name) # just the file name
print(csv_path.suffix) # the ending, like .csv
print(csv_path.exists()) # False for now, because we have not created the file yet
cities.csv
.csv
False
Path("gis_demo") is a relative path, meaning relative to the folder where the script runs. The / operator joins pieces, whatever the operating system. mkdir(exist_ok=True) creates the folder and does not complain if it already exists. .name and .suffix pick out the file name and its extension, and .exists() tells you whether the file is there.
2. Reading and writing text
notes = folder / "notes.txt"
# Step 1: write a text file. "w" = write (this replaces the file if it exists).
# "with" opens the file and closes it automatically when the indented block ends.
with open(notes, "w", encoding="utf-8") as f:
f.write("Bengaluru,12.9716,77.5946\n") # \n means "start a new line"
f.write("Mysuru,12.2958,76.6394\n")
# Step 2: read it back. "r" = read. Looping over the file gives one line at a time.
with open(notes, "r", encoding="utf-8") as f:
for line in f:
# strip() removes the \n at the end; split(",") cuts the line at each comma into 3 pieces
name, lat, lon = line.strip().split(",")
print(name, float(lat), float(lon)) # float() turns the text into a number
Bengaluru 12.9716 77.5946
Mysuru 12.2958 76.6394
"w" means write (replacing any old content), "r" means read and "a" means append. The with block closes the file for you when it ends, even if an error happens. Always use with. \n is a line break. The loop reads one line at a time, strip() removes the line break and split(",") cuts the line into three pieces that are unpacked into three variables.
3. CSV files
A CSV (comma separated values) file is a table in plain text. The csv module reads and writes it properly, including values that contain commas.
import csv # Python's built-in tool for CSV files
rows = [
{"city": "Bengaluru", "state": "Karnataka", "population": 8443675, "lat": 12.9716, "lon": 77.5946},
{"city": "Mysuru", "state": "Karnataka", "population": 920550, "lat": 12.2958, "lon": 76.6394},
{"city": "Chennai", "state": "Tamil Nadu", "population": 4646732, "lat": 13.0827, "lon": 80.2707},
]
# Step 1: write the rows to a CSV file. DictWriter writes dictionaries; fieldnames sets the column order.
with open(csv_path, "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=["city", "state", "population", "lat", "lon"])
writer.writeheader() # the first line with the column names
writer.writerows(rows) # one line per dictionary
# Step 2: read it back. DictReader gives each row as a dictionary keyed by the column names.
with open(csv_path, newline="", encoding="utf-8") as f:
for row in csv.DictReader(f):
# Note the type: everything read from a CSV is TEXT (str), even the population number
print(row["city"], row["population"], type(row["population"]))
Bengaluru 8443675 <class 'str'>
Mysuru 920550 <class 'str'>
Chennai 4646732 <class 'str'>
DictWriter turns each dictionary into a line, and DictReader gives each line back as a dictionary keyed by the header. Notice the type: everything read from a CSV is text. Convert with int() or float() yourself. Pandas does this automatically, which is a big reason to use it.
4. JSON files
JSON is text in the shape of Python dictionaries and lists. Web APIs and GeoJSON use it.
import json # JSON is the format web services and GeoJSON use
json_path = folder / "city.json"
city = {
"name": "Bengaluru",
"location": {"lat": 12.9716, "lon": 77.5946}, # a dictionary inside a dictionary
"tags": ["capital", "tech"], # a list inside a dictionary
}
# Step 1: save the dictionary as JSON text. indent=2 makes it easy to read.
with open(json_path, "w", encoding="utf-8") as f:
json.dump(city, f, indent=2)
# Step 2: read it back into a dictionary
with open(json_path, encoding="utf-8") as f:
data = json.load(f)
# Step 3: go down level by level to reach a value
print(data["name"])
print(data["location"]["lat"]) # first "location", then "lat" inside it
print(data["tags"][1]) # the second item (position 1) of the list
Bengaluru
12.9716
tech
json.dump writes and json.load reads. Nested values are reached by chaining square brackets: data["location"]["lat"] first takes the inner dictionary and then its lat key. A list of dictionaries, which is what most APIs return, converts directly to a pandas table in Part 7.
5. Dates and times
Satellite scenes, field surveys and sensor logs all carry dates. The datetime module handles them.
from datetime import date, datetime, timedelta
scene_date = date(2025, 3, 15) # a satellite image date: year, month, day
today = date(2025, 4, 10)
print(scene_date)
print((today - scene_date).days) # subtracting two dates gives a gap; .days is the number of days
print(scene_date + timedelta(days=16)) # timedelta is a length of time; add 16 days to get the next pass
print(scene_date.strftime("%d %B %Y")) # write the date your way: %d day, %B month name, %Y year
2025-03-15
26
2025-03-31
15 March 2025
Subtracting two dates gives a timedelta, and .days reads the number of days. Adding timedelta(days=16) moves forward one Landsat revisit cycle. strftime writes a date as text using codes: %d day, %m month number, %B month name, %Y four digit year.
Parse text into a date with strptime and the matching codes:
# strptime does the opposite of strftime: it reads text into a date, using the pattern you describe
stamp = datetime.strptime("2025-03-15 10:30", "%Y-%m-%d %H:%M")
print(stamp.year, stamp.month, stamp.hour)
# Four dates, 16 days apart, using a list comprehension (a loop written in one line)
dates = [date(2025, 3, 15) + timedelta(days=16 * i) for i in range(4)]
print([d.isoformat() for d in dates]) # isoformat gives the standard year-month-day text
2025 3 10
['2025-03-15', '2025-03-31', '2025-04-16', '2025-05-02']
The second part builds a sequence of dates 16 days apart. The ISO format YYYY-MM-DD sorts correctly as text and is the safest way to store dates.
6. Many files in a folder
# Step 1: create three small test files
for name in ["a_2025.csv", "b_2025.csv", "map.png"]:
(folder / name).write_text("demo")
# Step 2: glob finds files matching a pattern. * means "anything".
csv_files = sorted(folder.glob("*.csv"))
print([p.name for p in csv_files])
# Step 3: count the files by type. counts.get(key, 0) gives 0 the first time we see a type.
counts = {}
for p in folder.iterdir(): # every item in the folder
if p.is_file():
counts[p.suffix] = counts.get(p.suffix, 0) + 1
print(dict(sorted(counts.items())))
['a_2025.csv', 'b_2025.csv', 'cities.csv']
{'.csv': 3, '.json': 1, '.png': 1, '.txt': 1}
glob("*.csv") finds files matching a pattern. iterdir() lists everything in the folder, and is_file() skips sub-folders. counts.get(key, 0) + 1 is the standard way to count: start from 0 when the key is new. This is the heart of a GIS file inventory script. Use rglob instead of glob to search sub-folders too.
7. Modules and imports
A module is a .py file whose functions you can use from another file. import brings one in.
import math # Python's maths toolbox
from math import radians as to_radians # take one tool out and give it a shorter name
print(math.sqrt(144)) # square root
print(round(to_radians(180), 4)) # 180 degrees in radians (pi)
12.0
3.1416
import math gives access as math.sqrt. from ... import ... as ... takes one name and renames it. Aliases keep names short, and you will write import pandas as pd and import geopandas as gpd throughout this series.
Your own helper file is a module too. Create gis_utils.py next to your script:
# Your own functions can live in their own file and be imported, like any other tool.
# Step 1: write a small file called gis_utils.py (the text between the triple quotes becomes the file)
helper = '''
def clean_name(text):
return text.strip().title()
def valid_point(lat, lon):
return -90 <= lat <= 90 and -180 <= lon <= 180
'''
Path("gis_utils.py").write_text(helper)
# Step 2: make sure Python looks in the current folder for it
import sys
sys.path.insert(0, str(Path.cwd()))
# Step 3: import the file and use its functions
import gis_utils
print(gis_utils.clean_name(" new DELHI "))
print(gis_utils.valid_point(28.6, 77.2))
New Delhi
True
Here the file is written by code only so that this example is complete. Normally you create gis_utils.py in your editor and write just import gis_utils. The two sys.path lines make sure Python looks in the current folder. Put code in a module when you use it from more than one script.
8. Large files and generators
A list holds everything in memory. A generator makes one item at a time, using yield, so a huge file never has to fit in memory at once.
# A generator function uses "yield" to hand back one row at a time instead of loading the whole file.
# This matters when a file is too big to hold in memory.
def read_cities(path):
with open(path, newline="", encoding="utf-8") as f:
for row in csv.DictReader(f):
yield row["city"], int(row["population"]) # int() turns the text into a whole number
total = 0
for city, pop in read_cities(csv_path):
total += pop # += means "add to the existing value"
print(total)
14010957
read_cities is a generator function. Each time the loop asks for the next value, it runs until yield, hands back one record, and waits. For GIS this helps when a file has millions of points. Pandas can also read big files in chunks.
Run this part as a notebook
The code on this page is also a Jupyter notebook. It has the theory in text cells, then the code, then guided practice, an independent assignment and hidden solutions, so you can try each task before you look. Download the Part 5 notebook, open it in JupyterLab and run the cells from top to bottom. Part 1 shows how to install and start Jupyter.
Practice
- Write a list of five places to a CSV with
DictWriter, then read it back and print each population as anint. - Save a Python dictionary with a nested
locationto JSON and read one inner value. - Compute the number of days between two dates you choose and print one of them as
12 July 2024. - Count the files in a folder you choose by extension. Use
Path.home() / "Downloads"if you want a quick test. - Put two helper functions in your own
gis_utils.pyand import them.
Common mistakes
- Typing a Windows path with single backslashes. Use
Path("C:/data") / "file.csv"or a raw string. - Forgetting that CSV values are text until converted.
- Opening a file with
"w"when you meant"a", which erases the old content. - Forgetting
encoding="utf-8", which garbles place names with accents. - Giving your own file the name of a library, such as
csv.pyorjson.py. Python then imports yours. - Parsing a date with the wrong format codes.