Scripting¶

You can write Python scripts that call the GMrecordsApp application to create high-level workflows. In the example below we create a Python script to run several subcommands to download and process ground motions recorded close to the epicenter of a magnitude 4.5 earthquake, and then export the results to CSV files and generate a report summarizing the results. The configuration and parameter files are in the docs/contents/tutorials directory.

In this tutorial, the commands are written to be executed in Jupyter Notebooks. Within a Jupyter Notebook, commands can be redirected to the terminal with the ! symbol and we make ue of this functionality here.

Local `gmprocess` configuration¶

In this example we specify parameters in the project configuration to produce a small dataset. We use only the FDSN fetcher and limit the station distance to 0.1 degrees. The configuration files are in the conf/scripting directory.

First, we list the available projects in the current directory.

!gmrecords projects --list

INFO 2024-11-19 19:44:48 | gmrecords._initialize: Logging level includes INFO.
INFO 2024-11-19 19:44:48 | gmrecords._initialize: PROJECTS_PATH: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/.gmprocess
INFO 2024-11-19 19:44:48 | projects.main: Running subcommand 'projects'

Project: cli-tutorial **Current Project**
	Conf Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/conf/cli
	Data Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/cli


Project: scripting-tutorial 
	Conf Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/conf/scripting
	Data Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting

The PROJECTS_PATH shows the location of the projects configuration file where the information for the projects is stored. Second, we use the projects subcommand to select the project configuration scripting-tutorial.

!gmrecords projects --switch scripting-tutorial

INFO 2024-11-19 19:44:50 | gmrecords._initialize: Logging level includes INFO.
INFO 2024-11-19 19:44:50 | gmrecords._initialize: PROJECTS_PATH: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/.gmprocess
INFO 2024-11-19 19:44:50 | projects.main: Running subcommand 'projects'

Switched to project: 
Project: scripting-tutorial **Current Project**
	Conf Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/conf/scripting
	Data Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting

Third, we create the directory to hold the data.

Note

We include the directory with the processing parameters for the project in the source code.

At this point we have an empty data/scripting directory. The conf/scripting directory has two files: fetchers/yml and user.yml. These configuration files hold parameters that override default values provided with the source code. See Configuration File for more information.

Download Data¶

In this example we will consider ground motions recorded for a magnitude 4.5 earthquake east of San Francisco, California. We have cached a snippet of the results of running gmrecords download --eventid nc73291880 in the tests/data/tutorials directory. Consequently, we simply copy the data from tests/data/tutorials/nc73291880 to data/scripting/nc73291880.

!mkdir -p data/scripting/.
!cp -r ../../../tests/data/tutorials/nc73291880 data/scripting/.

List Data¶

We now have earthquake rupture information and raw waveforms in the data/scripting directory.

!tree data/scripting

data/scripting
└── nc73291880
    ├── event.json
    └── raw
        ├── BK.BRIB.01.HNE__20191015T053312Z__20191015T054042Z.mseed
        ├── BK.BRIB.01.HNN__20191015T053312Z__20191015T054042Z.mseed
        ├── BK.BRIB.01.HNZ__20191015T053312Z__20191015T054042Z.mseed
        └── BK.BRIB.xml

2 directories, 5 files

Python Script¶

First we need to import the GMrecordsApp application and initial it

from gmprocess.apps.gmrecords import GMrecordsApp

app = GMrecordsApp()
app.load_subcommands()

Now, we need to create a dictionary with the arguments common to all subcommands. We must include arguments that normally are given default values by the command line argument parser.

args = {
    'debug': False,
    'quiet': False,
    'event_id': "nc73291880",
    'textfile': None,
    'overwrite': False,
    'num_processes': 0,
    'label': None,
    'datadir': None,
    'confdir': None,
    'textfile': None,
    'resume': None,
}

And let’s make a convenience function to make calling the individual subcommands (i.e., “steps”) easier

def call_gmprocess_subcommand(subcommand):
    step_args = {
        'subcommand': subcommand,
        'func': app.classes[subcommand]['class'],
        'log': None,
    }
    args.update(step_args)
    app.main(**args)

Now we can easily call each subcommand

call_gmprocess_subcommand("assemble")

Show code cell output Hide code cell output

--------------------------------------------------------------------------------
Project: scripting-tutorial **Current Project**
	Conf Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/conf/scripting
	Data Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting
--------------------------------------------------------------------------------

INFO 2024-11-19 19:44:53 | gmrecords._initialize: Logging level includes INFO.
INFO 2024-11-19 19:44:53 | gmrecords._initialize: PROJECTS_PATH: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/.gmprocess
INFO 2024-11-19 19:44:53 | assemble.main: Running subcommand 'assemble'
INFO 2024-11-19 19:44:53 | assemble.main: Number of events to assemble: 1
INFO 2024-11-19 19:44:53 | assemble.main: Assembling event nc73291880 (1 of 1)...
INFO 2024-11-19 19:44:53 | assemble._assemble_event: Calling assemble function for nc73291880...
INFO 2024-11-19 19:44:55 | assemble_utils.load_rupture: Rupture geometry file not found.
INFO 2024-11-19 19:44:56 | assemble_utils.assemble: 
1 StationStreams(s) in StreamCollection:
  3 StationTrace(s) in StationStream (passed):
    BK.BRIB.01.HNE | 2019-10-15T05:33:12.810000Z - 2019-10-15T05:40:42.800000Z | 100.0 Hz, 45000 samples (passed)
    BK.BRIB.01.HNN | 2019-10-15T05:33:12.810000Z - 2019-10-15T05:40:42.800000Z | 100.0 Hz, 45000 samples (passed)
    BK.BRIB.01.HNZ | 2019-10-15T05:33:12.810000Z - 2019-10-15T05:40:42.800000Z | 100.0 Hz, 45000 samples (passed)
INFO 2024-11-19 19:44:56 | stream_workspace.add_streams: Adding waveforms for station BK.BRIB.HN
INFO 2024-11-19 19:44:56 | assemble._assemble_event: Done with assemble function for nc73291880...
INFO 2024-11-19 19:44:56 | base._summarize_files_created: The following files have been created:
INFO 2024-11-19 19:44:56 | base._summarize_files_created: File type: Workspace
INFO 2024-11-19 19:44:56 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/nc73291880/workspace.h5

call_gmprocess_subcommand("process_waveforms")

call_gmprocess_subcommand("compute_station_metrics")

call_gmprocess_subcommand("compute_waveform_metrics")

call_gmprocess_subcommand("export_metric_tables")

Show code cell output Hide code cell output

--------------------------------------------------------------------------------
Project: scripting-tutorial **Current Project**
	Conf Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/conf/scripting
	Data Path: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting
--------------------------------------------------------------------------------

INFO 2024-11-19 19:46:32 | gmrecords._initialize: Logging level includes INFO.
INFO 2024-11-19 19:46:32 | gmrecords._initialize: PROJECTS_PATH: /builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/.gmprocess
INFO 2024-11-19 19:46:32 | export_metric_tables.main: Running subcommand 'export_metric_tables'
INFO 2024-11-19 19:46:32 | export_metric_tables.main: Creating tables for event nc73291880 (1 of 1)...
INFO 2024-11-19 19:46:33 | base._summarize_files_created: The following files have been created:
INFO 2024-11-19 19:46:33 | base._summarize_files_created: File type: Metric tables
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_events.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_channels(component=h2).csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_rotd(percentile=50.0).csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_quadraticmean().csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_channels(component=z).csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_channels(component=h1).csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_geometricmean().csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_channels(component=h2)_README.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_rotd(percentile=50.0)_README.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_quadraticmean()_README.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_channels(component=z)_README.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_channels(component=h1)_README.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_metrics_geometricmean()_README.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_fit_spectra_parameters.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_fit_spectra_parameters_README.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_snr.csv
INFO 2024-11-19 19:46:33 | base._summarize_files_created: 	/builds/ghsc/esi/groundmotion-processing/docs/contents/tutorials/data/scripting/scripting-tutorial_default_snr_README.csv

call_gmprocess_subcommand("generate_station_maps")

This will produce CSV files with the waveform metrics in the data/scripting directory.

!ls -1 data/scripting/*.csv

data/scripting/scripting-tutorial_default_events.csv
data/scripting/scripting-tutorial_default_fit_spectra_parameters.csv
data/scripting/scripting-tutorial_default_fit_spectra_parameters_README.csv
'data/scripting/scripting-tutorial_default_metrics_channels(component=h1).csv'
'data/scripting/scripting-tutorial_default_metrics_channels(component=h1)_README.csv'
'data/scripting/scripting-tutorial_default_metrics_channels(component=h2).csv'
'data/scripting/scripting-tutorial_default_metrics_channels(component=h2)_README.csv'
'data/scripting/scripting-tutorial_default_metrics_channels(component=z).csv'
'data/scripting/scripting-tutorial_default_metrics_channels(component=z)_README.csv'
'data/scripting/scripting-tutorial_default_metrics_geometricmean().csv'
'data/scripting/scripting-tutorial_default_metrics_geometricmean()_README.csv'
'data/scripting/scripting-tutorial_default_metrics_quadraticmean().csv'
'data/scripting/scripting-tutorial_default_metrics_quadraticmean()_README.csv'
'data/scripting/scripting-tutorial_default_metrics_rotd(percentile=50.0).csv'
'data/scripting/scripting-tutorial_default_metrics_rotd(percentile=50.0)_README.csv'
data/scripting/scripting-tutorial_default_snr.csv
data/scripting/scripting-tutorial_default_snr_README.csv

The station map will be in the data/scripting/nc73291880 directory.

!ls -1 data/scripting/nc73291880/*.html

data/scripting/nc73291880/stations_map.html

Scripting¶

Local gmprocess configuration¶

Download Data¶

List Data¶

Python Script¶

Local `gmprocess` configuration¶