This guide has 2 routes. Pick one, or read both. Each route is complete on its own.
Weather stations are few, and their records have gaps. A reanalysis fills the world with numbers. It takes a weather forecasting model and, every few hours, adjusts it with all the observations that exist, from stations, balloons, ships and satellites. The result is a complete record of the atmosphere on a regular grid, for the past decades. ERA5, from the European Centre for Medium-Range Weather Forecasts, is the best known. It covers 1940 to the present, hourly, on a grid of about 30 km, for temperature, wind, humidity, pressure, rainfall and many other variables.
The original source of ERA5 is the Copernicus Climate Data Store. It gives you the complete data, but it needs an account and a key, and the files are large. For a point, there is an easier way: Open-Meteo is a free service that serves ERA5 values for any location through a web address. This guide uses it, and shows the original route as well.
Route A: the Copernicus Climate Data Store
The Climate Data Store (CDS) is the official source. The screens change from time to time, so check the labels against what you see.
- Open the dataset page ERA5 hourly data on single levels from 1940 to present. The page has the tabs Overview, Download, Analysis ready data and Documentation. A cookie window may ask for your choice first.
- Open the Download tab. You can look at the form without an account. The form has the fields Product type (Reanalysis, Ensemble members, Ensemble mean, Ensemble spread), Variable, Year, Month, Day and Time. Under Variable, a Popular group lists 2m temperature and Total precipitation, and the other groups include Precipitation and rain, Evaporation and runoff, Soil and Snow.
- Choose the Geographical area. The options are Whole available region, Area selection on a map, and Enter coordinates (bounding box) with North, South, West and East.
- Choose the Data format, GRIB or NetCDF4 (Experimental), and the Download format, Unarchived (not zipped if single file) or Zip.
- Under Terms of use you must accept the CC-BY licence before you can submit. The link reads Login/register to accept licences, so you need a free account for this step. Until you are logged in, the button at the bottom reads Login/register to submit request. After login it submits.
- The CDS prepares the file, and you download it when it is ready. For a long record or a large area, the request waits in a queue. Show API request code, on the same page, writes the request as Python code for the
cdsapipackage.
The labels in steps 1 to 5 were read on the live page without an account. What happens after login, in steps 5 and 6, was not run.
For use from Python, the cdsapi package sends the same request from a script. It needs the key from your account page, saved in a file in your home folder. This guide does not run it, because it needs your own key. The result of a CDS request is a gridded file for a box, which you open with xarray or rasterio. For a point or a few points, the route below is quicker.
Route B: Python, with Open-Meteo
The Open-Meteo historical weather API needs no account and no key. You give a latitude and a longitude and a period, and it answers with the values of the grid cell around the point. You need Python with requests and pandas.
import requests
import pandas as pd
# Step 1: the address of the service, and a name for your script
ARCHIVE = "https://archive-api.open-meteo.com/v1/archive"
HEADERS = {"User-Agent": "timeandspatial-guide/1.0 (write your email address here)"}
def daily(model, start, end, variables, latitude=30.79, longitude=75.47):
# Step 2: build the request. The model decides which dataset the numbers come from.
parameters = {
"latitude": latitude, "longitude": longitude, "start_date": start, "end_date": end,
"daily": ",".join(variables), "timezone": "Asia/Kolkata", "models": model,
}
reply = requests.get(ARCHIVE, params=parameters, headers=HEADERS, timeout=60)
reply.raise_for_status()
answer = reply.json()
# Step 3: the answer holds the grid cell that was used, its height, and the daily values
table = pd.DataFrame(answer["daily"])
table["time"] = pd.to_datetime(table["time"])
return table, answer
# Step 4: the daily rainfall and temperatures of 2023 at Jagraon, from ERA5
era5, answer = daily("era5", "2023-01-01", "2023-12-31", ["precipitation_sum", "temperature_2m_max", "temperature_2m_min"])
print("days:", len(era5))
print("point asked for: 30.79 N, 75.47 E | grid cell used:", answer["latitude"], "N,", answer["longitude"], "E | height of the cell:", answer["elevation"], "m")
print(era5.head(3).to_string(index=False))
days: 365
point asked for: 30.79 N, 75.47 E | grid cell used: 30.75 N, 75.5 E | height of the cell: 235.0 m
time precipitation_sum temperature_2m_max temperature_2m_min
2023-01-01 0.0 17.0 6.7
2023-01-02 0.0 15.1 4.5
2023-01-03 0.0 13.8 4.2
The service does not return the point you asked for. It returns the centre of the ERA5 grid cell that holds it, which here is at 30.75 N and 75.5 E, a few kilometres away, and it tells you so. The cell is about 30 km wide, so every point inside it gets the same numbers. The height of the cell, 235 m, is the average height of the model's land and may not match the height of your place.
Which model?
The request includes models. Without it, the service chooses and may mix datasets. Always say which model you want, so that you know what the numbers are. Compare what three choices return for July.
# Step 1: ask for July rainfall and temperature from three model settings
for model in ["era5", "era5_land", None]:
parameters = {"latitude": 30.79, "longitude": 75.47, "start_date": "2023-07-01", "end_date": "2023-07-31",
"daily": "precipitation_sum,temperature_2m_max", "timezone": "Asia/Kolkata"}
if model:
parameters["models"] = model
july = pd.DataFrame(requests.get(ARCHIVE, params=parameters, headers=HEADERS, timeout=60).json()["daily"])
label = model or "no model given"
print(f"{label:16s} rainfall values present: {int(july['precipitation_sum'].notna().sum()):2d} of 31 | July rainfall: {july['precipitation_sum'].sum():6.1f} mm | temperature values present: {int(july['temperature_2m_max'].notna().sum())}")
era5 rainfall values present: 31 of 31 | July rainfall: 199.7 mm | temperature values present: 31
era5_land rainfall values present: 0 of 31 | July rainfall: 0.0 mm | temperature values present: 31
no model given rainfall values present: 31 of 31 | July rainfall: 178.0 mm | temperature values present: 31
Three settings, three answers. ERA5 gives about 200 mm of rain in July, the default choice gives a smaller number, and ERA5-Land, a finer version of ERA5, returns no rainfall values for this request at all: all 31 are missing, and the sum shows 0. The temperatures are there. A sum of an empty column is zero, and zero rainfall is a wrong answer that raises no error. Always count the values that are present before you add them up.
Monthly rainfall and temperature
# Step 1: daily to monthly. Rainfall is added up, and the daily temperatures are averaged.
monthly = era5.groupby(era5["time"].dt.month).agg(
rain_mm=("precipitation_sum", "sum"),
mean_max_c=("temperature_2m_max", "mean"),
mean_min_c=("temperature_2m_min", "mean"),
).round(1)
monthly.index = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]
print(monthly.T.to_string())
# Step 2: the year in a few numbers
hottest = era5.loc[era5["temperature_2m_max"].idxmax()]
print("rainfall in 2023:", round(float(era5["precipitation_sum"].sum())), "mm")
print("hottest day:", hottest["time"].date(), "at", hottest["temperature_2m_max"], "degrees C")
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
rain_mm 37.4 1.5 65.7 42.2 77.6 48.7 199.7 25.3 83.5 6.0 21.1 0.0
mean_max_c 16.4 25.2 27.8 32.8 35.9 37.0 33.3 35.2 34.1 31.8 27.0 21.3
mean_min_c 6.6 11.5 16.2 18.9 22.3 26.6 27.4 27.7 26.0 19.4 14.5 8.7
rainfall in 2023: 609 mm
hottest day: 2023-05-22 at 43.7 degrees C
The pattern is the climate of the Punjab plain: hot summers with a maximum of 36 to 37 degrees in May and June, cool winters with minima of 7 to 9 degrees, and rain that peaks with the monsoon in July.
Compare with CHIRPS
The CHIRPS guide built the monthly rainfall of 2023 for the same place from satellite and station data. Put the two side by side.
# Step 1: the monthly CHIRPS rainfall of 2023 at Jagraon, from the CHIRPS guide
chirps = [28.0, 11.1, 60.6, 13.5, 55.5, 42.5, 275.7, 115.7, 201.1, 40.8, 11.5, 4.4]
# Step 2: the two datasets in one table, with the difference
comparison = pd.DataFrame({"ERA5 mm": monthly["rain_mm"].values, "CHIRPS mm": chirps}, index=monthly.index)
comparison["difference mm"] = (comparison["ERA5 mm"] - comparison["CHIRPS mm"]).round(1)
print(comparison.T.to_string())
print("total: ERA5", round(float(comparison["ERA5 mm"].sum())), "mm | CHIRPS", round(float(comparison["CHIRPS mm"].sum())), "mm")
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
ERA5 mm 37.4 1.5 65.7 42.2 77.6 48.7 199.7 25.3 83.5 6.0 21.1 0.0
CHIRPS mm 28.0 11.1 60.6 13.5 55.5 42.5 275.7 115.7 201.1 40.8 11.5 4.4
difference mm 9.4 -9.6 5.1 28.7 22.1 6.2 -76.0 -90.4 -117.6 -34.8 9.6 -4.4
total: ERA5 609 mm | CHIRPS 860 mm
The two datasets disagree more than you might expect. CHIRPS is higher over the year, by roughly 250 mm. The biggest gap is in September, 118 mm, and August follows, where ERA5 has about a fifth of the CHIRPS value. The datasets are made differently. ERA5 is a model that estimates rain from the state of the atmosphere, on a 30 km grid, and rainfall is its weakest variable. CHIRPS combines satellite estimates with station records on a 5 km grid. Neither is the truth. Compare with a rain gauge if you have one, and say which dataset you used.
Save the data
# Step 1: write the daily table, with the settings in the file name
era5.to_csv("jagraon_era5_daily_2023.csv", index=False)
# Step 2: read it back to check
check = pd.read_csv("jagraon_era5_daily_2023.csv", parse_dates=["time"])
print("saved:", len(check), "days |", check["time"].min().date(), "to", check["time"].max().date(), "| columns:", check.columns.tolist())
saved: 365 days | 2023-01-01 to 2023-12-31 | columns: ['time', 'precipitation_sum', 'temperature_2m_max', 'temperature_2m_min']
Terms and credit
Open-Meteo states its terms on its website, and the terms for free use differ from those for commercial use, so read them. Open-Meteo asks for a credit line, and ERA5 has its own: the Copernicus Climate Change Service asks users to cite ERA5 and to state that the data contains modified Copernicus Climate Change Service information. The paper in the sources is the one to cite. The service also limits how many requests you may send, so ask for a year of one point, as here, and not a request for each day.
What a reanalysis cannot tell you
- It is a model, not a measurement. A weather station beside your field will be closer to the truth for that field.
- The grid is 30 km. Local storms, hills and a town's heat island vanish.
- Rainfall is the weakest variable, especially in the tropics and with convective rain. Temperature is usually better.
- The height of the grid cell may differ from the height of your place, and temperature falls by about 6.5 degrees for every 1000 m of height.
- The daily totals follow the time zone you ask for. Use the local one, as here, or the day will be cut at the wrong hour.
Common mistakes
- Adding up a column that is empty and reading the zero as no rain.
- Not choosing the model, and not knowing which dataset the numbers come from.
- Treating the returned grid cell as your point.
- Comparing ERA5 rain with a rain gauge as if they measured the same thing.
- Requesting a year of data one day at a time, and being stopped by the limit.
- Leaving out the credit lines.