Ocelote Functions

A complete reference guide for using ocelote within a Python session.

Commands to run hazard assessments, and upload results to USGS assets.

Function

Description

authenticate s3

Acquires, tests, and stores credentials for uploading assessments to S3

authenticate sciencebase

Acquires, tests, and stores credentials for uploading assessments to ScienceBase

initialize fire

Initializes a folder for a new fire event

initialize version

Initializes a folder for a new assessment version

download

Downloads assessment datasets from the internet

preprocess

Preprocesses datasets in preparation for an assessment

run

Runs assessment and exports results to comply with the PWFDF data spec

upload s3

Uploads assessment results to S3 for the web map

upload sciencebase

Uploads assessment results to ScienceBase for permanent archival

version

Returns the version string of the currently installed ocelote package
ocelote.authenticate_s3(*, reset=False, check=True)

Acquires, tests, and stores credentials for uploading assessments to S3

Authenticate
authenticate_s3()

Prompts the user for credentials needed to upload assessments to S3. Tests the credentials can be used to login, then saves to ~/.ocelote/credentials.json for later uploads.

Reset
authenticate_s3(reset=True)

Deletes all stored S3 credentials.

Skip Login
authenticate_s3(check=False)

Does not test that credentials can be used to login to S3. User provided values are accepted as-is, and saved directly to ~/.ocelote/credentials.json.

Inputs:

  • reset bool – True to delete all stored credentials. False (default) to prompt the user for new credentials.

  • check bool – True (default) to test credentials before saving. False to save credentials without testing their ability to login.

ocelote.authenticate_sciencebase(*, reset: bool = False, check: bool = True)

Acquires, tests, and stores credentials for uploading assessments to ScienceBase

Authenticate
authenticate_sciencebase()

Prompts the user for credentials needed to upload assessments to ScienceBase. Tests the credentials can be used to login, then saves to ~/.ocelote/credentials.json for later uploads. Note that the user’s AD password is never saved - instead, it is only used to test the user’s ability to log in.

Reset
authenticate_sciencebase(reset=True)

Deletes all stored ScienceBase credentials.

Skip Login
authenticate_sciencebase(check=False)

Does not test that credentials can be used to login to ScienceBase. User provided values are accepted as-is, and saved directly to ~/.ocelote/credentials.json.

Inputs:

  • reset bool – True to delete all stored credentials. False (default) to prompt the user for new credentials.

  • check bool – True (default) to test credentials before saving. False to save credentials without testing their ability to login.

ocelote.initialize_fire(path, parent=None)

Initializes a folder for a new fire

Initialize Fire
initialize_fire(path)

Initializes a new fire folder at the indicated path. The new folder will include a “fire.py” metadata file and an empty “inputs” folder. If the input path is a relative path, then the path is parsed relative to the current folder. If the path already exists, then it must be empty. Returns the path to the initialized folder as output.

Custom Path
initialize_fire(path, parent)

Parses relative paths relative to the “parent” folder, rather than the current folder. The “parent” input should be a path, and is typically an absolute path. But if not, then the parent path is parsed relative to the current folder.

Inputs:

  • path Path | str – The path to the new fire folder

  • parent Path | str – The anchor folder for parsing relative paths. Defaults to the current folder.

Outputs:

Path – The path to the initialized fire folder

ocelote.initialize_version(version, fire=None, *, source=None, inherit=True)

Initializes the folder for a new assessment version

Initialize Version
initialize_version(version)
initialize_version(version, fire)

Initializes a folder for a new assessment version. The input version string must follow the format X.Y where X and Y are both integers. The name of the new folder will match this version string, and the folder will include version.py, datasets.py, and configuration.py config files.

By default, initializes the new version in the current folder. Alternatively, use the fire option to specify the path to a fire folder in which to initialize the new version. Returns the path to the initialized folder as output.

By default, if the fire folder contains existing versions, then the command will inherit datasets.py and configuration.py metadata from the latest version that does not exceed the new version number. Read the the syntax below for additional metadata inheritance options.

From Source
initialize_version(..., *, source)

Often, input datasets for an assessment version are provided by a known source. These include BAER, GTAC, and EROS. Use the source option to indicate that datasets were provided by one of these sources. This will auto-fill relevant “source” fields in the initialized datasets.py with a characteristic string for the provider. Supported sources include: baer, eros, and gtac.

Inherit Metadata
initialize_version(..., *, inherit)

Specifies options for inheriting metadata from previously existing versions. The inherit option should be a bool, string, or Path. If False, then the command will not inherit metadata, and the new datasets.py` and configuration.py` files will be set to their defaults. If True, then the command will attempt to detect whether their are any previously existing versions. If there are existing versions, then the command will inherit metadata from the latest version that does not exceed the new version number.

Alternatively, inherit` can be a string or Path indicating a specific version folder from which to inherit metadata. If a string or relative Path, then the path is parsed relative to the fire folder. Raises an error if the indicated version does not have the expected metadata files

Inputs:

  • version str | Path – The version string for the new assessment version

  • fire str | Path – The path to a fire folder

  • source “baer” | “eros” | “gtac” – A data provider for the assessment. Must be baer, eros, or gtac

  • inherit bool | str | Path – Specifies how to inherit metadata. False to not inherit, True to auto-detect existing versions, or a string/Path to inherit from a specific folder.

Outputs:

Path – The path to the initialized assessment version folder

ocelote.download(version=None, fire=None, *, atlas14=True)

Downloads datasets for the indicated assessment version

Download Data
download()
download(version)
download(version, fire)

Downloads datasets for the indicated assessment version. Reads the datasets.py configuration file, and determines any datasets that should be downloaded. Buffers the fire perimeter, and uses the buffered bounding box to download datasets within its extent. Downloaded datasets are saved in the inputs folder for the assessment version. Also downloads NOAA Atlas 14 data (in addition to the other datasets) - the downloaded NOAA Atlas 14 data is for the coordinate at the center of the analysis domain.

The version input should be the name or path to an assessment version. If you do not provide a version, then interprets the current folder as the assessment version. If version is a name or relative path, then it is interpreted relative to the current folder. Alternatively, use the fire input to parse the version path relative to a given fire folder path. The fire input is usually an absolute path, but if it is also a relative path, then it is parsed relative to the current folder.

Returns a dict whose keys are the names of the datasets that were download. The key values are the paths to the downloaded datasetes.

Skip NOAA Atlas 14
download(..., *, atlas14=False)

Does not download data from NOAA Atlas 14.

Inputs:

  • version str | Path – The name or path of the assessment version whose datasets should be downloaded

  • fire str | Path – The path to a fire folder that should be the anchor for parsing relative assessment version paths

  • atlas14 bool – True (default) to download NOAA Atlas 14 data. False to not download this data.

Outputs:

dict – Keys are the names of downloaded datasets, values are the paths of the downloaded files

ocelote.preprocess(version=None, fire=None)

Preprocesses datasets for an assessment version

preprocess(version)
preprocess(version, fire)

Uses wildcat to preprocesses the input datasets for an assessment version. Reprojects datasets to the same CRS, and clips everything to match the bounds of the buffered burn perimeter. Optionally estimates burn severity and terrain masks, and ensures dNBR and KF-factor datasets are within valid data ranges. The wildcat settings are determined by the configuration.py metadata file. You may use any config settings for wildcat, with the exception of input datasets, which will be ignored. You can read more about wildcat preprocessor settings in the wildcat docs: https://ghsc.code-pages.usgs.gov/lhp/wildcat/api/config/preprocess.html

The version input should be the name or path to an assessment version. If you do not provide a version, then interprets the current folder as the assessment version. If version is a name or relative path, then it is interpreted relative to the current folder. Alternatively, use the fire input to parse the version path relative to a given fire folder path. The fire input is usually an absolute path, but if it is also a relative path, then it is parsed relative to the current folder. Returns the path to the folder holding the preprocessed datasets as output.

You should run the download command before this command. Uses datasets.py to determine datasets that are expected to be downloaded. Raises an error if the file for a downloaded dataset is missing.

Inputs:

  • version str | Path – The name or path of an assessment version that should be preprocessed

  • fire str | Path – The path to a fire folder used as the anchor for parsing relative assessment version paths

Outputs:

Path – The path to the folder holding preprocessed results

ocelote.run(version=None, fire=None, *, ignore_name=False, ignore_tiles=False, ignore_excluded=False)

Runs an assessment and packages the results for a ScienceBase upload

Run Assessment
run(version)
run(version, fire)

Uses wildcat to run and export an assessment. Validates assessment metadata, delineates a stream network, runs hazard models, and exports the results to various file formats. Exported results will conform to the PWFDF data spec. This command will produce a results folder, and an results.zip file, both of which contain the exported assessment results. The results folder is intended for use in quality control, and users may inspect its contents as they like. The results.zip file holds archived results for ScienceBase and should not be opened or altered. The command also produces a slim <fire id>.zip archive suitable for distribution via email. This archive is intended as a backup in case results cannot be distributed via ScienceBase.

The version input should be the name or path to an assessment version. If you do not provide a version, then interprets the current folder as the assessment version. If version is a name or relative path, then it is interpreted relative to the current folder. Alternatively, use the fire input to parse the version path relative to a given fire folder path. The fire input is usually an absolute path, but if it is also a relative path, then it is parsed relative to the current folder.

Returns a dict with two keys results and upload. The value for each key is the Path to the associated set of results.

Ignore Metadata
run(..., ignore_name = True)
run(..., ignore_tiles = True)
run(..., ignore_excluded = True)

Options to ignore certain metadata fields when validating the datasets.py configuration file. Use ignore_name to not require the fire name to use standard naming conventions. Use ignore_tiles to not require dem.tiles metadata when the DEM is not downloaded. Use ignore_excluded to allow excluded.source metadata to use non-standard text.

Inputs:

  • version str | Path – The name or path of an assessment version that should be run

  • fire str | Path – The path to a fire folder used as the anchor for parsing relative assessment version paths

  • ignore_name bool – Whether to ignore fire names that don’t follow standard naming conventions

  • ignore_tiles bool – Whether to ignore missing DEM tile metadata

  • ignore_excluded bool – Whether to ignore non-standard exclusion mask source metadata

Outputs:

dict – Keys are results and upload. Values are the paths to the associated resources

ocelote.upload_s3(version=None, fire=None, *, region_name=None, bucket_name=None, aws_access_key_id=None, aws_secret_access_key=None)

Uploads assessment results to S3

Upload to S3
upload_s3(version)
upload_s3(version, fire)

Uploads assessment results to S3, which is used to implement web mapping services. By default, uses S3 credentials obtained by the authenticate s3 command, but see below to pass credentials as parameters.

The version input should be the name or path to an assessment version. If you do not provide a version, then interprets the current folder as the assessment version. If version is a name or relative path, then it is interpreted relative to the current folder. Alternatively, use the fire input to parse the version path relative to a given fire folder path. The fire input is usually an absolute path, but if it is also a relative path, then it is parsed relative to the current folder.

Override Credentials
upload_s3(..., *, region_name)
upload_s3(..., *, bucket_name)
upload_s3(..., *, aws_access_key_id)
upload_s3(..., *, aws_secret_access_key)

Specifies the credentials used to upload to S3. Any credentials provided as keywords will override credentials stored by the authenticate s3 command.

Inputs:

  • version str | Path – The name or path of an assessment version that should be run

  • fire str | Path – The path to a fire folder used as the anchor for parsing relative assessment version paths

  • region_name str – The name of the AWS region with the S3 bucket

  • bucket_name str – The name of the S3 bucket

  • aws_access_key_id str – An AWS access key ID

  • aws_secret_access_key str – An AWS secret access key

ocelote.upload_sciencebase(version=None, fire=None, *, user=None, password=None, parent=None)

Uploads assessment results to S3

Upload to ScienceBase
upload_sciencebase(version)
upload_sciencebase(version, fire)

Uploads assessment results to ScienceBase. By default, uses ScienceBase credentials obtained by the authenticate sciencebase command, but see below to pass credentials as parameters.

The version input should be the name or path to an assessment version. If you do not provide a version, then interprets the current folder as the assessment version. If version is a name or relative path, then it is interpreted relative to the current folder. Alternatively, use the fire input to parse the version path relative to a given fire folder path. The fire input is usually an absolute path, but if it is also a relative path, then it is parsed relative to the current folder.

Override Credentials
upload_sciencebase(..., *, user)
upload_sciencebase(..., *, password)

Specifies the credentials used to upload to ScienceBase. Any credentials provided as keywords will override credentials stored by the authenticate sciencebase command. Note that the user input should be a complete AD email address (including the @usgs.gov or other suffix).

Parent Collection
upload_sciencebase(..., *, parent)

Uploads results to the indicated parent item. By default, the command uploads assessments to the live PWFDF collection (parent item 6818f950d4be0208bc3e0165). Use the parent input to upload results to a different parent item instead. This can be useful for uploading assessments to a mock collection for testing purposes.

Inputs:

  • version str | Path – The name or path of an assessment version that should be run

  • fire str | Path – The path to a fire folder used as the anchor for parsing relative assessment version paths

  • user str – The AD user email address

  • password str – The AD password

  • parent str – The ID of the ScienceBase parent collection at which to upload results

ocelote.version()

Returns the version string of the currently installed ocelote package

version()

Returns the version string of the currently installed ocelote package.

Outputs:

str – The current ocelote version string