Download Assessments

Many users will want to use the pwfdf-api to download assessment datasets. You can do so using the download command. In brief, this command will search the PWFDF collection, and download matching assessments onto the local filesystem. Each downloaded assessment will be placed in a subfolder matching the assessment’s name. The command also includes options to:

  • Select which assessments are downloaded,

  • Select which assessment files to download,

  • Indicate where to download assessments, and

  • Limit the size of downloaded resources

Selecting Assessments

You can select which assessments should be downloaded using any of the search filters discussed in the search guide. For example:

pwfdf download --fire Eaton --max-fires 1
pwfdf download --extent California --start-date 2025-01

You can select which assessments should be downloaded using any of the search filters discussed in the search guide. For example:

from pwfdf_api import download

download(fire="Eaton")
download(extent="California", start_date="2025-01")

Folder Paths

The command will download files for each assessment in the search results, and the files for each assessment will be placed in a subfolder matching the assessment name. By default, the assessment subfolders are created in the current folder. Use the --parent option to create the subfolders in a different folder instead:

pwfdf download --parent path/to/my/folder

By default, a subfolder’s name may contain spaces and punctuation. Use the --normalize option to replaces spaces with underscores, and puncutation with hyphens:

pwfdf download --normalize

By default, the command will not overwrite data, so will raise an error if an assessment subfolder already exists. Use the --duplicates option to take other actions when this occurs. Options include:

Duplicates Action

Description

skip

Does not download the assessment, and continues to the next assessment.

replace

Downloads the assessment, overwriting any content in the existing folder.

error (default)

Raises an error

For example, to skip assessments that have already been downloaded:

pwfdf download --duplicates skip

The command will download files for each assessment in the search results, and the files for each assessment will be placed in a subfolder matching the assessment name. By default, the assessment subfolders are created in the current folder. Use parent to create the subfolders in a different folder instead:

download(parent="path/to/my/folder")

By default, a subfolder’s name may contain spaces and punctuation. Set normalize=True to replace spaces with underscores, and puncutation with hyphens:

download(normalize=True)

By default, the command will not overwrite data, so will raise an error if an assessment subfolder already exists. Use duplicates to specify other actions when this occurs. Options include:

Duplicates Action

Description

skip

Does not download the assessment, and continues to the next assessment.

replace

Downloads the assessment, overwriting any content in the existing folder.

error (default)

Raises an error

For example, to skip assessments that have already been downloaded:

download(duplicates="skip")

Downloaded Files

By default, downloads all files associated with each assessment. These include:

Name

Description

Shapefile.zip

Hazard assessment results in a Shapefile format

GeoJSON.zip

Hazard assessment results in a GeoJSON format

median-thresholds.csv

Table of median rainfall thresholds as calculated over the stream segment network and catchment basins.

metadata.json

JSON file with metadata about an assessment’s fire event, implementation, hazard models, and input datasets.

inputs.zip

Zip archive of hazard assessment input datasets that are not accessible via a separate DOI.

configuration.txt

Configuration settings used by the wildcat package to implement the hazard assessment.

<fire id>.xml

Assessment metadata used to implement the ScienceBase interface.

Use the --files option to select specific files to download. The file selectors are the same as the names in the above table, with the exception of the xml metadata, which can be selected using fgdc-metadata.xml:

# Downloads shapefiles, median thresholds, and metadata.json
pwfdf download --files Shapefile.zip median-thresholds.csv metadata.json

By default, the command will unzip any downloaded zip archives. Use the --no-extract to leave the archives zipped:

pwfdf download --no-extract

Use files to select specific files to download. The file selectors are the same as the names in the above table, with the exception of the xml metadata, which can be selected using fgdc-metadata.xml:

# Downloads shapefiles, median thresholds, and metadata.json
files = ["Shapefile.zip", "median-thresholds.csv", "metadata.json"]
download(files=files)

By default, the command will unzip any downloaded zip archives. Set extract=False to leave the archives zipped:

download(extract=False)

Maximum Size

By default, the command will raise an error if the downloaded files would exceed 2 GB. Use the --max-gb option to change this limit:

pwfdf download --max-gb 3

Caution

The --max-gb option limits the amount of data transferred over an internet connection, but the size of the download on the local filesystem may still exceed this limit. This will occur if you download any zip archives and the archives are extracted (the default behavior). Use --no-extract if you want --max-gb to also limit the size on the local filesystem:

# Raises an error if the download would exceed 3 GB on the local filesystem
pwfdf download --max-gb 3 --no-extract

By default, the command will raise an error if the downloaded files would exceed 2 GB. Use max_gb to change this limit:

download(max_gb=3)

Caution

The max_gb option limits the amount of data transferred over an internet connection, but the size of the download on the local filesystem may still exceed this limit. This will occur if you download any zip archives and the archives are extracted (the default behavior). Set extract=False if you want max_gb to also limit the size on the local filesystem:

# Raises an error if the download would exceed 3 GB on the local filesystem
download(max_gb=3, extract=False)

Dry Run

In some cases, you may want to check what the download command would do, without actually downloading any data. This can be useful for verifying the assessments or total size of the download. You can run the command without downloading data using the --dry-run option. The effects of the command can then be determined by inspecting the command logs. For example:

$ pwfdf download --extent California --start-date 2025-01 --dry-run
Searching the PWFDF collection
Found 6 matching product records
Estimated size of unzipped, downloaded files: 0.01505 GB
Downloading 6 assessments

Note that you can use the --verbose option to print more detailed logs:

$ pwfdf download --extent California --start-date 2025-01 --dry-run --verbose
Searching the PWFDF collection
Found 6 matching product records
Estimated size of unzipped, downloaded files: 0.01505 GB
Downloading 6 assessments
    Downloading Border 2 2025-01-23 v1.0
    Downloading Eaton 2025-01-08 v1.0
    Downloading Hughes 2025-01-22 v1.0
    Downloading Hurst 2025-01-07 v1.0
    Downloading Kenneth 2025-01-09 v1.0
    Downloading Palisades 2025-01-07 v1.0

Tip

You can save a verbose log to file using the --log option:

pwfdf download --extent California --dry-run --log my-log.txt

In some cases, you may want to check what the download command would do, without actually downloading any data. This can be useful for verifying the assessments or total size of the download. You can run the command without downloading data by setting dry_run = True. The effects of the command can then be determined by inspecting the command logs. Note that you can use the pwfdf_api.logging package to configure console and/or file logs. For example:

>>> import pwfdf_api as pwfdf

>>> # Configure a DEBUG-level console or file logger
>>> pwfdf.logging.console.verbose()
>>> pwfdf.logging.file.verbose('my-log.txt')

>>> # Do a download dry run and inspect the logs
>>> download(extent="California", start_date="2025-01", dry_run=True)
Searching the PWFDF collection
Found 6 matching product records
Estimated size of unzipped, downloaded files: 0.01505 GB
Downloading 6 assessments
    Downloading Border 2 2025-01-23 v1.0
    Downloading Eaton 2025-01-08 v1.0
    Downloading Hughes 2025-01-22 v1.0
    Downloading Hurst 2025-01-07 v1.0
    Downloading Kenneth 2025-01-09 v1.0
    Downloading Palisades 2025-01-07 v1.0