NOAA Data Source

This notebooks shows how to use retrieve data from NOAA, there are two ways of doing so:

• By FTP: It’s the easier way, but it will download the bulk of the data(3Gb+), so it may take some time.

• By API: It’s the cleaner way, as it provides more granular access.

Please note that in order to use the API, some required reference files will be downloaded too.

Using the API

To use the API, you just need to pass the list of countries and the start and end dates, you can see an example below:

from datetime import datetime
from task_geo.data_sources.noaa import noaa_api

start_date = datetime(2020, 1, 1)
end_date = datetime(2020, 1, 15)
countries = ['FR']

data = noaa_api(countries, start_date, end_date)

2020-04-03 18:17:07,112 - INFO - ftp_connector - Connecting to NOAA FTP server.
2020-04-03 18:17:59,726 - INFO - api_connector - Requesting data for FR

type(data)

pandas.core.frame.DataFrame

data.head()

	latitude	longitude	elevation	country	name	date	station	tmax	tmin
0	48.0689	-1.7339	36.0	France	RENNES-ST JACQUES	2020-01-01	FR000007130	10.4	4.8
1	48.0689	-1.7339	36.0	France	RENNES-ST JACQUES	2020-01-02	FR000007130	11.0	7.8
2	48.0689	-1.7339	36.0	France	RENNES-ST JACQUES	2020-01-03	FR000007130	13.1	NaN
3	48.0689	-1.7339	36.0	France	RENNES-ST JACQUES	2020-01-04	FR000007130	10.4	1.4
4	48.0689	-1.7339	36.0	France	RENNES-ST JACQUES	2020-01-05	FR000007130	9.5	3.0

Using the FTP

To download the bulk of the files you just need to run the code in the cell below, it will download the files and extract the compressed ones:

from ftp import download_noaa_files, process_noaa_data

download_noaa_files()

Now that we have downloaded the files, in order to load the data from our local disk and retrieve a pandas.DataFrame with data, we just need to pass it to process_noaa_files with a list of FIPS codes for the countries we want data from.

For more info FIPS codes, visit the wikipedia page: https://en.wikipedia.org/wiki/List_of_FIPS_country_codes

countries = ['FR'] # We use the FIPS code to identify the countries
data = process_noaa_data(countries)